[Spacewalk-list] Spacewalk performance tuning for deployments with 1000+ hosts

Wed Oct 29 14:49:09 UTC 2014

Does this solve problem with web ui ?

Sent from my Windows Phone

-----Original Message-----
From: "Amedeo Salvati" <amedeo at oscert.net>
Sent: ‎2014-‎10-‎29 12:03
To: "spacewalk-list at redhat.com" <spacewalk-list at redhat.com>
Cc: "spacewalk-list at redhat.com" <spacewalk-list at redhat.com>
Subject: Re: [Spacewalk-list]Spacewalk performance tuning for deployments with 1000+ hosts

repeat I think you must use rhn|spacewalk proxies

best regards
a

Da: spacewalk-list-bounces at redhat.com
A: spacewalk-list at redhat.com
Cc: 
Data: Wed, 29 Oct 2014 07:22:30 +0100
Oggetto: Re: [Spacewalk-list] Spacewalk performance tuning for deployments with 1000+ hosts

> Unfortunately it doesn't help :(

> When i want do sth with all 1500 systems i get :

> Service Temporarily Unavailable
> The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

> 2014-10-28 17:41 GMT+01:00 Matthew Madey <mattmadey at gmail.com>:

> Here are some configurations you might find helpful for tuning Apache\Tomcat\Java\Networking.. Like others have mentioned.. when you get over 1000+ clients, it's a good idea to start scaling horizontally with Spacewalk Proxies. We use 4 proxies in our production environment and are servicing 8000+ clients. We can actually patch 1600 clients at a time and the GUI is still pretty responsive. Can't guarantee this will resolve your issue, but this worked for us. 

> Add maxThreads to /etc/tomcat6/server.xml
>  
>     <Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" URIEncoding="UTF-8" address="127.0.0.1" maxThreads="1024" maxKeepAliveRequests="1000"/>>     <!-- A "Connector" using the shared thread pool-->>     <!-->     <Connector executor="tomcatThreadPool">                port="8080" protocol="HTTP/1.1" >                connectionTimeout="20000" >                redirectPort="8443" />>  >     <!-- Define an AJP 1.3 Connector on port 8009 -->>     <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" URIEncoding="UTF-8" address="127.0.0.1" maxThreads="1024"/>>   >     <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" URIEncoding="UTF-8" address="::1" maxThreads="1024"/>>  >   > Tune Apache to service more requests /etc/httpd/conf.d/zz-spacewalk-server.conf>  >  > #######################################################> # Authorship and versioning info> # $Author$> # $Date$> # $URL$> # $Rev$> # deployment_location: /etc/httpd/conf.d/> #######################################################> # ** DO NOT EDIT **> # Master configuration file for the rhn_server setup> #>  > ##> ## Spacewalk settings> ##>  > <VirtualHost *>>  > <IfModule mod_jk.c>>     # Inherit the mod_jk settings defined in zz-spacewalk-www.conf>     JkMountCopy On> </IfModule>>  > <Directory "/var/www/html/*">>         AllowOverride all> </Directory>>  > RewriteEngine on> RewriteOptions inherit> </VirtualHost>>  > # Override default httpd prefork settings> <IfModule prefork.c>> StartServers       8> MinSpareServers    400> MaxSpareServers   400> ServerLimit      1024> MaxClients       1024> MaxRequestsPerChild  200> </IfModule>>  > Include /etc/rhn/satellite-httpd/conf/rhn/rhn_monitoring.conf>  >  >  >  Also added some network tuning to /etc/sysctl.conf>  > net.ipv4.icmp_echo_ignore_broadcasts = 1> net.ipv4.conf.all.secure_redirects = 0> net.ipv4.tcp_max_syn_backlog = 8192> net.ipv4.conf.default.secure_redirects = 0> net.ipv4.tcp_syncookies = 1> net.ipv4.conf.all.accept_source_route = 0> net.ipv4.conf.all.rp_filter = 1> net.ipv4.conf.all.send_redirects = 0> net.ipv4.conf.default.accept_redirects = 0> net.ipv4.conf.all.accept_redirects = 0> net.ipv4.conf.default.send_redirects = 0> net.core.somaxconn = 1536> net.core.dev_weight = 512> ##3x normal for a queue and budget suited to networks greater than 100mbps> net.core.netdev_budget = 10000> net.core.netdev_max_backlog = 30000> Depending on the amount of memory on your Spacewalk server, you may want to increase your JAVA_OPTS Xms and Xmx settings to something a little higher. Typically only needed if you are seeing Java Heap out of memory errors in your Spacewalk logs.

> On Tue, Oct 28, 2014 at 11:21 AM, Waldirio Manhães Pinheiro <waldirio at gmail.com> wrote:

> Dear Krzysztof

> Have you checked your numa configuration ?!

> Maybe you can customize you environment to use the same bus to application / memory.

> ______________
> Atenciosamente
> Waldirio
> msn: waldirio at gmail.com
> Skype: waldirio
> Site: www.waldirio.com.br
> Blog: blog.waldirio.com.br
> LinkedIn: http://br.linkedin.com/pub/waldirio-pinheiro/22/b21/646
> PGP: www.waldirio.com.br/public.html

> On Tue, Oct 28, 2014 at 2:10 PM, Krzysztof Pawłowski <msciciel at msciciel.eu> wrote:

> We have dedicated machine for db pgsql (16GB RAM , 8 cores)  and seperate for spacewalk (16GB RAM, 8 Cores).

> I think that problem is with enormous number of queries to database. During such request db is not utilized 100% and spacewalk is also not 100% utilized.

> 2014-10-28 16:25 GMT+01:00 Götz Reinicke - IT Koordinator <goetz.reinicke at filmakademie.de>:

> Hi,
> Am 28.10.14 um 12:57 schrieb Krzysztof Pawłowski:

> > Hi,
> > Is there any guide about tuning spacewalk performance ? With every new
> > host spacewalk is getting slowly. Using SSM with more than 200-300 hosts
> > is impossible due timeouts. It's also not possible to deploy config
> > files to all hosts.
> > Standard java tuning was done, java gc is not the problem now.
> >
> > Any suggestions ?

> what server hardware do you use? What is the systemload while performing
> that tasks? CPU, RAM, Disksystem, IO, Network .... ?

>         /Götz

> --
> Götz Reinicke
> IT-Koordinator

> Tel. +49 7141 969 82 420
> E-Mail goetz.reinicke at filmakademie.de

> Filmakademie Baden-Württemberg GmbH
> Akademiehof 10
> 71638 Ludwigsburg
> www.filmakademie.de

> Eintragung Amtsgericht Stuttgart HRB 205016

> Vorsitzender des Aufsichtsrats: Jürgen Walter MdL

[The entire original message is not included.]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20141029/49625b91/attachment.htm>