[Spacewalk-list] Spacewalk performance tuning for deployments with 1000+ hosts

Krzysztof Pawłowski msciciel at msciciel.eu
Fri Nov 21 05:46:42 UTC 2014


Hello Mathew,
Could You provide also configuration parameters for java vm for tomcat and
rhn-search ?

2014-10-28 17:41 GMT+01:00 Matthew Madey <mattmadey at gmail.com>:

>
> Here are some configurations you might find helpful for tuning Apache\Tomcat\Java\Networking..
> Like others have mentioned.. when you get over 1000+ clients, it's a good
> idea to start scaling horizontally with Spacewalk Proxies. We use 4 proxies
> in our production environment and are servicing 8000+ clients. We can
> actually patch 1600 clients at a time and the GUI is still pretty
> responsive. Can't guarantee this will resolve your issue, but this worked
> for us.
>
> Add maxThreads to /etc/tomcat6/server.xml
>
>
>
>     <Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" URIEncoding="UTF-8" address="127.0.0.1" maxThreads="1024" maxKeepAliveRequests="1000"/>
>
>     <!-- A "Connector" using the shared thread pool-->
>
>     <!--
>
>     <Connector executor="tomcatThreadPool"
>
>                port="8080" protocol="HTTP/1.1"
>
>                connectionTimeout="20000"
>
>                redirectPort="8443" />
>
>
>
>     <!-- Define an AJP 1.3 Connector on port 8009 -->
>
>     <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" URIEncoding="UTF-8" address="127.0.0.1" maxThreads="1024"/>
>
>
>
>     <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" URIEncoding="UTF-8" address="::1" maxThreads="1024"/>
>
>
>
>
>
> Tune Apache to service more requests /etc/httpd/conf.d/zz-spacewalk-server.conf
>
>
>
>
>
> #######################################################
>
> # Authorship and versioning info
>
> # $Author$
>
> # $Date$
>
> # $URL$
>
> # $Rev$
>
> # deployment_location: /etc/httpd/conf.d/
>
> #######################################################
>
> # ** DO NOT EDIT **
>
> # Master configuration file for the rhn_server setup
>
> #
>
>
>
> ##
>
> ## Spacewalk settings
>
> ##
>
>
>
> <VirtualHost *>
>
>
>
> <IfModule mod_jk.c>
>
>     # Inherit the mod_jk settings defined in zz-spacewalk-www.conf
>
>     JkMountCopy On
>
> </IfModule>
>
>
>
> <Directory "/var/www/html/*">
>
>         AllowOverride all
>
> </Directory>
>
>
>
> RewriteEngine on
>
> RewriteOptions inherit
>
> </VirtualHost>
>
>
>
> # Override default httpd prefork settings
>
> <IfModule prefork.c>
>
> StartServers       8
>
> MinSpareServers    400
>
> MaxSpareServers   400
>
> ServerLimit      1024
>
> MaxClients       1024
>
> MaxRequestsPerChild  200
>
> </IfModule>
>
>
>
> Include /etc/rhn/satellite-httpd/conf/rhn/rhn_monitoring.conf
>
>
>
>
>
>
>
>  Also added some network tuning to /etc/sysctl.conf
>
>
>
> net.ipv4.icmp_echo_ignore_broadcasts = 1
>
> net.ipv4.conf.all.secure_redirects = 0
>
> net.ipv4.tcp_max_syn_backlog = 8192
>
> net.ipv4.conf.default.secure_redirects = 0
>
> net.ipv4.tcp_syncookies = 1
>
> net.ipv4.conf.all.accept_source_route = 0
>
> net.ipv4.conf.all.rp_filter = 1
>
> net.ipv4.conf.all.send_redirects = 0
>
> net.ipv4.conf.default.accept_redirects = 0
>
> net.ipv4.conf.all.accept_redirects = 0
>
> net.ipv4.conf.default.send_redirects = 0
>
> net.core.somaxconn = 1536
>
> net.core.dev_weight = 512
>
> ##3x normal for a queue and budget suited to networks greater than 100mbps
>
> net.core.netdev_budget = 10000
>
> net.core.netdev_max_backlog = 30000
>
>
>
> Depending on the amount of memory on your Spacewalk server, you may want to increase your JAVA_OPTS Xms and Xmx settings to something a little higher. Typically only needed if you are seeing Java Heap out of memory errors in your Spacewalk logs.
>
>
>
> On Tue, Oct 28, 2014 at 11:21 AM, Waldirio Manhães Pinheiro <
> waldirio at gmail.com> wrote:
>
>> Dear Krzysztof
>>
>> Have you checked your numa configuration ?!
>>
>> Maybe you can customize you environment to use the same bus to
>> application / memory.
>>
>> ______________
>> Atenciosamente
>> Waldirio
>> msn: waldirio at gmail.com
>> Skype: waldirio
>> Site: www.waldirio.com.br
>> Blog: blog.waldirio.com.br
>> LinkedIn: http://br.linkedin.com/pub/waldirio-pinheiro/22/b21/646
>> PGP: www.waldirio.com.br/public.html
>>
>> On Tue, Oct 28, 2014 at 2:10 PM, Krzysztof Pawłowski <
>> msciciel at msciciel.eu> wrote:
>>
>>> We have dedicated machine for db pgsql (16GB RAM , 8 cores)  and
>>> seperate for spacewalk (16GB RAM, 8 Cores).
>>>
>>> I think that problem is with enormous number of queries to database.
>>> During such request db is not utilized 100% and spacewalk is also not 100%
>>> utilized.
>>>
>>>
>>>
>>> 2014-10-28 16:25 GMT+01:00 Götz Reinicke - IT Koordinator <
>>> goetz.reinicke at filmakademie.de>:
>>>
>>>> Hi,
>>>> Am 28.10.14 um 12:57 schrieb Krzysztof Pawłowski:
>>>> > Hi,
>>>> > Is there any guide about tuning spacewalk performance ? With every new
>>>> > host spacewalk is getting slowly. Using SSM with more than 200-300
>>>> hosts
>>>> > is impossible due timeouts. It's also not possible to deploy config
>>>> > files to all hosts.
>>>> > Standard java tuning was done, java gc is not the problem now.
>>>> >
>>>> > Any suggestions ?
>>>>
>>>> what server hardware do you use? What is the systemload while performing
>>>> that tasks? CPU, RAM, Disksystem, IO, Network .... ?
>>>>
>>>>
>>>>         /Götz
>>>>
>>>> --
>>>> Götz Reinicke
>>>> IT-Koordinator
>>>>
>>>> Tel. +49 7141 969 82 420
>>>> E-Mail goetz.reinicke at filmakademie.de
>>>>
>>>> Filmakademie Baden-Württemberg GmbH
>>>> Akademiehof 10
>>>> 71638 Ludwigsburg
>>>> www.filmakademie.de
>>>>
>>>> Eintragung Amtsgericht Stuttgart HRB 205016
>>>>
>>>> Vorsitzender des Aufsichtsrats: Jürgen Walter MdL
>>>> Staatssekretär im Ministerium für Wissenschaft,
>>>> Forschung und Kunst Baden-Württemberg
>>>>
>>>> Geschäftsführer: Prof. Thomas Schadt
>>>>
>>>>
>>>> _______________________________________________
>>>> Spacewalk-list mailing list
>>>> Spacewalk-list at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>>
>>>
>>>
>>> _______________________________________________
>>> Spacewalk-list mailing list
>>> Spacewalk-list at redhat.com
>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>
>>
>>
>> _______________________________________________
>> Spacewalk-list mailing list
>> Spacewalk-list at redhat.com
>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>
>
>
> _______________________________________________
> Spacewalk-list mailing list
> Spacewalk-list at redhat.com
> https://www.redhat.com/mailman/listinfo/spacewalk-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20141121/1bf3aea8/attachment.htm>


More information about the Spacewalk-list mailing list