<div dir="ltr"><div>Unfortunately it doesn't help :( </div>When i want do sth with all 1500 systems i get : <div><h1>Service Temporarily Unavailable</h1> The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later. </div></div><div class="gmail_extra"> <div class="gmail_quote">2014-10-28 17:41 GMT+01:00 Matthew Madey <<a href="mailto:mattmadey@gmail.com" target="_blank">mattmadey@gmail.com</a>>: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div> </div>Here are some configurations you might find helpful for tuning Apache\Tomcat\Java\Networking.. Like others have mentioned.. when you get over 1000+ clients, it's a good idea to start scaling horizontally with Spacewalk Proxies. We use 4 proxies in our production environment and are servicing 8000+ clients. We can actually patch 1600 clients at a time and the GUI is still pretty responsive. Can't guarantee this will resolve your issue, but this worked for us. <div> </div><div>Add maxThreads to /etc/tomcat6/server.xml<div style="color:rgb(80,0,80);font-family:arial,sans-serif;font-size:13px"> </div><div style="font-family:arial,sans-serif;font-size:13px"><pre style="white-space:pre-wrap;color:rgb(80,0,80);word-wrap:break-word"> <Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" URIEncoding="UTF-8" address="127.0.0.1" maxThreads="1024" maxKeepAliveRequests="1000"/></pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)"> </pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)"> </pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)"> <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" URIEncoding="UTF-8" address="127.0.0.1" maxThreads="1024"/></pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)"> </pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)"> <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" URIEncoding="UTF-8" address="::1" maxThreads="1024"/></pre><div style="color:rgb(80,0,80)"><pre style="white-space:pre-wrap"> </pre></div><div style="color:rgb(80,0,80)"><pre style="white-space:pre-wrap"> </pre></div><div style="color:rgb(80,0,80)"><pre style="white-space:pre-wrap">Tune Apache to service more requests /etc/httpd/conf.d/zz-spacewalk-server.conf</pre></div><div style="color:rgb(80,0,80)"><pre style="white-space:pre-wrap"> </pre></div><div style="color:rgb(80,0,80)"><pre style="white-space:pre-wrap"> </pre></div><div style="color:rgb(80,0,80)"><pre style="white-space:pre-wrap;word-wrap:break-word">#######################################################</pre><pre style="white-space:pre-wrap"># Authorship and versioning info</pre><pre style="white-space:pre-wrap"># $Author$</pre><pre style="white-space:pre-wrap"># $Date$</pre><pre style="white-space:pre-wrap"># $URL$</pre><pre style="white-space:pre-wrap"># $Rev$</pre><pre style="white-space:pre-wrap"># deployment_location: /etc/httpd/conf.d/</pre><pre style="white-space:pre-wrap">#######################################################</pre><pre style="white-space:pre-wrap"># ** DO NOT EDIT **</pre><pre style="white-space:pre-wrap"># Master configuration file for the rhn_server setup</pre><pre style="white-space:pre-wrap">#</pre><pre style="white-space:pre-wrap"> </pre><pre style="white-space:pre-wrap">##</pre><pre style="white-space:pre-wrap">## Spacewalk settings</pre><pre style="white-space:pre-wrap">##</pre><pre style="white-space:pre-wrap"> </pre><pre style="white-space:pre-wrap"><VirtualHost *></pre><pre style="white-space:pre-wrap"> </pre><pre style="white-space:pre-wrap"><IfModule mod_jk.c></pre><pre style="white-space:pre-wrap"> # Inherit the mod_jk settings defined in zz-spacewalk-www.conf</pre><pre style="white-space:pre-wrap"> JkMountCopy On</pre><pre style="white-space:pre-wrap"></IfModule></pre><pre style="white-space:pre-wrap"> </pre><pre style="white-space:pre-wrap"><Directory "/var/www/html/*"></pre><pre style="white-space:pre-wrap"> AllowOverride all</pre><pre style="white-space:pre-wrap"></Directory></pre><pre style="white-space:pre-wrap"> </pre><pre style="white-space:pre-wrap">RewriteEngine on</pre><pre style="white-space:pre-wrap">RewriteOptions inherit</pre><pre style="white-space:pre-wrap"></VirtualHost></pre><pre style="white-space:pre-wrap"> </pre><pre style="white-space:pre-wrap"># Override default httpd prefork settings</pre><pre style="white-space:pre-wrap"><IfModule prefork.c></pre><pre style="white-space:pre-wrap">StartServers 8</pre><pre style="white-space:pre-wrap">MinSpareServers 400</pre><pre style="white-space:pre-wrap">MaxSpareServers 400</pre><pre style="white-space:pre-wrap">ServerLimit 1024</pre><pre style="white-space:pre-wrap">MaxClients 1024</pre><pre style="white-space:pre-wrap">MaxRequestsPerChild 200</pre><pre style="white-space:pre-wrap"></IfModule></pre><pre style="white-space:pre-wrap"> </pre><pre style="white-space:pre-wrap">Include /etc/rhn/satellite-httpd/conf/rhn/rhn_monitoring.conf</pre><div><pre style="white-space:pre-wrap"> </pre></div><div><pre style="white-space:pre-wrap"> </pre></div><div><pre style="white-space:pre-wrap"> </pre></div></div><div style="color:rgb(80,0,80)"><pre style="white-space:pre-wrap"> Also added some network tuning to /etc/sysctl.conf</pre></div><div style="color:rgb(80,0,80)"><pre style="white-space:pre-wrap"> </pre></div><div><pre style="white-space:pre-wrap;color:rgb(80,0,80)">net.ipv4.icmp_echo_ignore_broadcasts = 1</pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)">net.ipv4.conf.all.secure_redirects = 0</pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)">net.ipv4.tcp_max_syn_backlog = 8192</pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)">net.ipv4.conf.default.secure_redirects = 0</pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)">net.ipv4.tcp_syncookies = 1</pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)">net.ipv4.conf.all.accept_source_route = 0</pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)">net.ipv4.conf.all.rp_filter = 1</pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)">net.ipv4.conf.all.send_redirects = 0</pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)">net.ipv4.conf.default.accept_redirects = 0</pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)">net.ipv4.conf.all.accept_redirects = 0</pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)">net.ipv4.conf.default.send_redirects = 0</pre><pre style="white-space:pre-wrap;color:rgb(80,0,80);word-wrap:break-word">net.core.somaxconn = 1536</pre><pre style="white-space:pre-wrap;color:rgb(80,0,80);word-wrap:break-word">net.core.dev_weight = 512</pre><pre style="white-space:pre-wrap;color:rgb(80,0,80);word-wrap:break-word">##3x normal for a queue and budget suited to networks greater than 100mbps</pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)">net.core.netdev_budget = 10000</pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)">net.core.netdev_max_backlog = 30000</pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)"> </pre><pre style="white-space:pre-wrap;color:rgb(80,0,80)"> </pre><pre style="white-space:pre-wrap">Depending on the amount of memory on your Spacewalk server, you may want to increase your JAVA_OPTS Xms and Xmx settings to something a little higher. Typically only needed if you are seeing Java Heap out of memory errors in your Spacewalk logs.</pre></div><div> </div></div></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"> <div class="gmail_quote">On Tue, Oct 28, 2014 at 11:21 AM, Waldirio Manhães Pinheiro <<a href="mailto:waldirio@gmail.com" target="_blank">waldirio@gmail.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Dear Krzysztof<div> </div><div>Have you checked your numa configuration ?!</div><div> </div><div>Maybe you can customize you environment to use the same bus to application / memory.</div></div><div class="gmail_extra"> <div><div>______________ Atenciosamente Waldirio msn: <a href="mailto:waldirio@gmail.com" target="_blank">waldirio@gmail.com</a> Skype: waldirio Site: <a href="http://www.waldirio.com.br" target="_blank">www.waldirio.com.br</a> Blog: <a href="http://blog.waldirio.com.br" target="_blank">blog.waldirio.com.br</a></div> <div>LinkedIn: <a href="http://br.linkedin.com/pub/waldirio-pinheiro/22/b21/646" target="_blank">http://br.linkedin.com/pub/waldirio-pinheiro/22/b21/646</a> PGP: <a href="http://www.waldirio.com.br/public.html" target="_blank">www.waldirio.com.br/public.html</a></div></div><div><div> <div class="gmail_quote">On Tue, Oct 28, 2014 at 2:10 PM, Krzysztof Pawłowski <<a href="mailto:msciciel@msciciel.eu" target="_blank">msciciel@msciciel.eu</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>We have dedicated machine for db pgsql (16GB RAM , 8 cores) and seperate for spacewalk (16GB RAM, 8 Cores). </div> </div>I think that problem is with enormous number of queries to database. During such request db is not utilized 100% and spacewalk is also not 100% utilized. <div><div> </div></div></div><div class="gmail_extra"> <div class="gmail_quote"><div><div>2014-10-28 16:25 GMT+01:00 Götz Reinicke - IT Koordinator <<a href="mailto:goetz.reinicke@filmakademie.de" target="_blank">goetz.reinicke@filmakademie.de</a>>: </div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>Hi, Am 28.10.14 um 12:57 schrieb Krzysztof Pawłowski: <div><div>> Hi, > Is there any guide about tuning spacewalk performance ? With every new > host spacewalk is getting slowly. Using SSM with more than 200-300 hosts > is impossible due timeouts. It's also not possible to deploy config > files to all hosts. > Standard java tuning was done, java gc is not the problem now. > > Any suggestions ? </div></div>what server hardware do you use? What is the systemload while performing that tasks? CPU, RAM, Disksystem, IO, Network .... ? /Götz -- Götz Reinicke IT-Koordinator Tel. <a href="tel:%2B49%207141%20969%2082%20420" value="+49714196982420" target="_blank">+49 7141 969 82 420</a> E-Mail <a href="mailto:goetz.reinicke@filmakademie.de" target="_blank">goetz.reinicke@filmakademie.de</a> Filmakademie Baden-Württemberg GmbH Akademiehof 10 71638 Ludwigsburg <a href="http://www.filmakademie.de" target="_blank">www.filmakademie.de</a> Eintragung Amtsgericht Stuttgart HRB 205016 Vorsitzender des Aufsichtsrats: Jürgen Walter MdL Staatssekretär im Ministerium für Wissenschaft, Forschung und Kunst Baden-Württemberg Geschäftsführer: Prof. Thomas Schadt </div></div>_______________________________________________ Spacewalk-list mailing list <a href="mailto:Spacewalk-list@redhat.com" target="_blank">Spacewalk-list@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/spacewalk-list" target="_blank">https://www.redhat.com/mailman/listinfo/spacewalk-list</a> </blockquote></div> </div> _______________________________________________ Spacewalk-list mailing list <a href="mailto:Spacewalk-list@redhat.com" target="_blank">Spacewalk-list@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/spacewalk-list" target="_blank">https://www.redhat.com/mailman/listinfo/spacewalk-list</a> </blockquote></div> </div></div></div> _______________________________________________ Spacewalk-list mailing list <a href="mailto:Spacewalk-list@redhat.com" target="_blank">Spacewalk-list@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/spacewalk-list" target="_blank">https://www.redhat.com/mailman/listinfo/spacewalk-list</a> </blockquote></div> </div> </div></div> _______________________________________________ Spacewalk-list mailing list <a href="mailto:Spacewalk-list@redhat.com">Spacewalk-list@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/spacewalk-list" target="_blank">https://www.redhat.com/mailman/listinfo/spacewalk-list</a> </blockquote></div> </div>