<div class="xam_msg_class"> <div style="font: normal 13px Arial; color:rgb(31, 28, 27);">this will scale out (not scale UP) your env, but you will change where your client will be connected -> to spacewalk or to spacewalk proxy<br><br> <div><span style="font-family:Arial; font-size:11px; color:#5F5F5F;">Da</span><span style="font-family:Arial; font-size:12px; color:#5F5F5F; padding-left:5px;">: spacewalk-list-bounces@redhat.com</span></div> <div><span style="font-family:Arial; font-size:11px; color:#5F5F5F;">A</span><span style="font-family:Arial; font-size:12px; color:#5F5F5F; padding-left:5px;">: spacewalk-list@redhat.com</span></div> <div><span style="font-family:Arial; font-size:11px; color:#5F5F5F;">Cc</span><span style="font-family:Arial; font-size:12px; color:#5F5F5F; padding-left:5px;">: </span></div> <div><span style="font-family:Arial; font-size:11px; color:#5F5F5F;">Data</span><span style="font-family:Arial; font-size:12px; color:#5F5F5F; padding-left:5px;">: Wed, 29 Oct 2014 15:49:09 +0100</span></div> <div><span style="font-family:Arial; font-size:11px; color:#5F5F5F;">Oggetto</span><span style="font-family:Arial; font-size:12px; color:#5F5F5F; padding-left:5px;">: Re: [Spacewalk-list] Spacewalk performance tuning for deployments with 1000+ hosts</span></div> <br> <div class="xam_msg_class"> <meta http-equiv="Content-Type" content="text/html; charset=utf-8"><div><div style="font-family: Calibri,sans-serif; font-size: 11pt;">> Does this solve problem with web ui ?<br><br>> Sent from my Windows Phone</div></div><div dir="ltr"><hr><span style="font-family: Calibri,sans-serif; font-size: 11pt; font-weight: bold;">> From: </span><span style="font-family: Calibri,sans-serif; font-size: 11pt;"><a href="mailto:amedeo@oscert.net" _djrealurl="mailto:amedeo@oscert.net">Amedeo Salvati</a></span><br><span style="font-family: Calibri,sans-serif; font-size: 11pt; font-weight: bold;">> Sent: </span><span style="font-family: Calibri,sans-serif; font-size: 11pt;">‎2014-‎10-‎29 12:03</span><br><span style="font-family: Calibri,sans-serif; font-size: 11pt; font-weight: bold;">> To: </span><span style="font-family: Calibri,sans-serif; font-size: 11pt;"><a href="mailto:spacewalk-list@redhat.com" _djrealurl="mailto:spacewalk-list@redhat.com">spacewalk-list@redhat.com</a></span><br><span style="font-family: Calibri,sans-serif; font-size: 11pt; font-weight: bold;">> Cc: </span><span style="font-family: Calibri,sans-serif; font-size: 11pt;"><a href="mailto:spacewalk-list@redhat.com" _djrealurl="mailto:spacewalk-list@redhat.com">spacewalk-list@redhat.com</a></span><br><span style="font-family: Calibri,sans-serif; font-size: 11pt; font-weight: bold;">> Subject: </span><span style="font-family: Calibri,sans-serif; font-size: 11pt;">Re: [Spacewalk-list]Spacewalk performance tuning for deployments with 1000+ hosts</span><br><br></div><div class="xam_msg_class"> <div style="font: 13px/normal Arial; color: rgb(31, 28, 27); font-size-adjust: none; font-stretch: normal;">> repeat I think you must use rhn|spacewalk proxies<br><br>> best regards<br>> a<br><br><br> <div><span style="color: rgb(95, 95, 95); font-family: Arial; font-size: 11px;">> Da</span><span style="color: rgb(95, 95, 95); padding-left: 5px; font-family: Arial; font-size: 12px;">: spacewalk-list-bounces@redhat.com</span></div> <div><span style="color: rgb(95, 95, 95); font-family: Arial; font-size: 11px;">> A</span><span style="color: rgb(95, 95, 95); padding-left: 5px; font-family: Arial; font-size: 12px;">: spacewalk-list@redhat.com</span></div> <div><span style="color: rgb(95, 95, 95); font-family: Arial; font-size: 11px;">> Cc</span><span style="color: rgb(95, 95, 95); padding-left: 5px; font-family: Arial; font-size: 12px;">: </span></div> <div><span style="color: rgb(95, 95, 95); font-family: Arial; font-size: 11px;">> Data</span><span style="color: rgb(95, 95, 95); padding-left: 5px; font-family: Arial; font-size: 12px;">: Wed, 29 Oct 2014 07:22:30 +0100</span></div> <div><span style="color: rgb(95, 95, 95); font-family: Arial; font-size: 11px;">> Oggetto</span><span style="color: rgb(95, 95, 95); padding-left: 5px; font-family: Arial; font-size: 12px;">: Re: [Spacewalk-list] Spacewalk performance tuning for deployments with 1000+ hosts</span></div> <br> <div class="xam_msg_class"> <div dir="ltr"><div>> > Unfortunately it doesn't help :(<br><br></div>> > When i want do sth with all 1500 systems i get :<br><div><h1>> > Service Temporarily Unavailable</h1> <p>> > The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.</p><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">> > 2014-10-28 17:41 GMT+01:00 Matthew Madey <span dir="ltr"><<a href="mailto:mattmadey@gmail.com" _djrealurl="mailto:mattmadey@gmail.com" target="_blank">mattmadey@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left-color: rgb(204, 204, 204); border-left-width: 1px; border-left-style: solid;"><div dir="ltr"><div><br></div>> > Here are some configurations you might find helpful for <span>tuning</span> Apache\Tomcat\Java\Networking.. Like others have mentioned.. when you get over 1000+ clients, it's a good idea to start scaling horizontally with Spacewalk Proxies. We use 4 proxies in our production environment and are servicing 8000+ clients. We can actually patch 1600 clients at a time and the GUI is still pretty responsive. Can't guarantee this will resolve your issue, but this worked for us. <div><br></div><div><p class="MsoNormal" style="color: rgb(80, 0, 80); font-family: arial,sans-serif; font-size: 13px;">> > Add maxThreads to /etc/tomcat6/server.xml<u></u><u></u></p><div style="color: rgb(80, 0, 80); font-family: arial,sans-serif; font-size: 13px;"><p class="MsoNormal"><u></u>> > <u></u></p></div><div style="font-family: arial,sans-serif; font-size: 13px;"><pre style="color: rgb(80, 0, 80); white-space: pre-wrap; -ms-word-wrap: break-word;"><span style="color: black;">> > <Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8443" URIEncoding="UTF-8" address="127.0.0.1" maxThreads="1024" maxKeepAliveRequests="1000"/><u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > <u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > <u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" URIEncoding="UTF-8" address="127.0.0.1" maxThreads="1024"/><u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > <u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" URIEncoding="UTF-8" address="::1" maxThreads="1024"/><u></u><u></u></span></pre><div style="color: rgb(80, 0, 80);"><pre style="white-space: pre-wrap;"><span style="color: black;"><u></u>> > <u></u></span></pre></div><div style="color: rgb(80, 0, 80);"><pre style="white-space: pre-wrap;"><span style="color: black;"><u></u>> > </span><span style="color: black; font-family: arial,sans-serif;"> </span></pre></div><div style="color: rgb(80, 0, 80);"><pre style="white-space: pre-wrap;"><span style="color: black;"><span>> > Tune</span> Apache to service more requests /etc/httpd/conf.d/zz-spacewalk-server.conf</span><u></u><u></u></pre></div><div style="color: rgb(80, 0, 80);"><pre style="white-space: pre-wrap;"><u></u>> > <u></u></pre></div><div style="color: rgb(80, 0, 80);"><pre style="white-space: pre-wrap;"><u></u>> > <u></u></pre></div><div style="color: rgb(80, 0, 80);"><pre style="white-space: pre-wrap; -ms-word-wrap: break-word;"><span style="color: black;">> > #######################################################<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > # Authorship and versioning info<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > # $Author$<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > # $Date$<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > # $URL$<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > # $Rev$<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > # deployment_location: /etc/httpd/conf.d/<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > #######################################################<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > # ** DO NOT EDIT **<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > # Master configuration file for the rhn_server setup<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > #<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;"><u></u>> > <u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > ##<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > ## Spacewalk settings<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > ##<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;"><u></u>> > <u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > <VirtualHost *><u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;"><u></u>> > <u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > <IfModule mod_jk.c><u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > # Inherit the mod_jk settings defined in zz-spacewalk-www.conf<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > JkMountCopy On<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > </IfModule><u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;"><u></u>> > <u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > <Directory "/var/www/html/*"><u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > AllowOverride all<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > </Directory><u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;"><u></u>> > <u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > RewriteEngine on<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > RewriteOptions inherit<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > </VirtualHost><u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;"><u></u>> > <u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > # Override default httpd prefork settings<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > <IfModule prefork.c><u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > StartServers 8<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > MinSpareServers 400<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > MaxSpareServers 400<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > ServerLimit 1024<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > MaxClients 1024<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > MaxRequestsPerChild 200<u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > </IfModule><u></u><u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;"><u></u>> > <u></u></span></pre><pre style="white-space: pre-wrap;"><span style="color: black;">> > Include /etc/rhn/satellite-httpd/conf/rhn/rhn_monitoring.conf<u></u><u></u></span></pre><div><pre style="white-space: pre-wrap;"><u></u>> > <u></u></pre></div><div><pre style="white-space: pre-wrap;"><u></u>> > <u></u></pre></div><div><pre style="white-space: pre-wrap;"><u></u>> > <u></u></pre></div></div><div style="color: rgb(80, 0, 80);"><pre style="white-space: pre-wrap;"><span style="color: black;"><u></u>> > </span><span style="color: black; font-family: arial,sans-serif;">Also added some network <span>tuning</span> to /etc/sysctl.conf</span></pre></div><div style="color: rgb(80, 0, 80);"><pre style="white-space: pre-wrap;"><span style="color: black;"><u></u>> > <u></u></span></pre></div><div><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > net.ipv4.icmp_echo_ignore_broadcasts = 1<u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > net.ipv4.conf.all.secure_redirects = 0<u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > net.ipv4.tcp_max_syn_backlog = 8192<u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > net.ipv4.conf.default.secure_redirects = 0<u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > net.ipv4.tcp_syncookies = 1<u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > net.ipv4.conf.all.accept_source_route = 0<u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > net.ipv4.conf.all.rp_filter = 1<u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > net.ipv4.conf.all.send_redirects = 0<u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > net.ipv4.conf.default.accept_redirects = 0<u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > net.ipv4.conf.all.accept_redirects = 0<u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > net.ipv4.conf.default.send_redirects = 0<u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap; -ms-word-wrap: break-word;"><span style="color: black;">> > net.core.somaxconn = 1536<u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap; -ms-word-wrap: break-word;"><span style="color: black;">> > net.core.dev_weight = 512<u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap; -ms-word-wrap: break-word;"><span style="color: black;">> > ##3x normal for a queue and budget suited to networks greater than 100mbps<u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > net.core.netdev_budget = 10000<u></u><u></u></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;">> > net.core.netdev_max_backlog = 30000</span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;"><br></span></pre><pre style="color: rgb(80, 0, 80); white-space: pre-wrap;"><span style="color: black;"><br></span></pre><pre style="white-space: pre-wrap;"><font color="#000000">> > Depending on the amount of memory on your Spacewalk server, you may want to increase your JAVA_OPTS Xms and Xmx settings to something a little higher. Typically only needed if you are seeing Java Heap out of memory errors in your Spacewalk logs.</font></pre></div><div><font color="#000000"><br></font></div></div></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">> > On Tue, Oct 28, 2014 at 11:21 AM, Waldirio Manhães Pinheiro <span dir="ltr"><<a href="mailto:waldirio@gmail.com" _djrealurl="mailto:waldirio@gmail.com" target="_blank">waldirio@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left-color: rgb(204, 204, 204); border-left-width: 1px; border-left-style: solid;"><div dir="ltr">> > Dear Krzysztof<div><br></div><div>> > Have you checked your numa configuration ?!</div><div><br></div><div>> > Maybe you can customize you environment to use the same bus to application / memory.</div></div><div class="gmail_extra"><br clear="all"><div><div>> > ______________<br>> > Atenciosamente<br>> > Waldirio<br>> > msn: <a href="mailto:waldirio@gmail.com" _djrealurl="mailto:waldirio@gmail.com" target="_blank">waldirio@gmail.com</a><br>> > Skype: waldirio<br>> > Site: <a href="http://www.waldirio.com.br" _djrealurl="http://www.waldirio.com.br" target="_blank">www.waldirio.com.br</a><br>> > Blog: <a href="http://blog.waldirio.com.br" _djrealurl="http://blog.waldirio.com.br" target="_blank">blog.waldirio.com.br</a></div> <div>> > LinkedIn: <a href="http://br.linkedin.com/pub/waldirio-pinheiro/22/b21/646" _djrealurl="http://br.linkedin.com/pub/waldirio-pinheiro/22/b21/646" target="_blank">http://br.linkedin.com/pub/waldirio-pinheiro/22/b21/646</a><br>> > PGP: <a href="http://www.waldirio.com.br/public.html" _djrealurl="http://www.waldirio.com.br/public.html" target="_blank">www.waldirio.com.br/public.html</a></div></div><div><div> <br><div class="gmail_quote">> > On Tue, Oct 28, 2014 at 2:10 PM, Krzysztof Pawłowski <span dir="ltr"><<a href="mailto:msciciel@msciciel.eu" _djrealurl="mailto:msciciel@msciciel.eu" target="_blank">msciciel@msciciel.eu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left-color: rgb(204, 204, 204); border-left-width: 1px; border-left-style: solid;"><div dir="ltr"><div><div>> > We have dedicated machine for db pgsql (16GB RAM , 8 cores) and seperate for spacewalk (16GB RAM, 8 Cores).<br></div><br></div>> > I think that problem is with enormous number of queries to database. During such request db is not utilized 100% and spacewalk is also not 100% utilized.<br><div><div><br><br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div>> > 2014-10-28 16:25 GMT+01:00 Götz Reinicke - IT Koordinator <span dir="ltr"><<a href="mailto:goetz.reinicke@filmakademie.de" _djrealurl="mailto:goetz.reinicke@filmakademie.de" target="_blank">goetz.reinicke@filmakademie.de</a>></span>:<br></div></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left-color: rgb(204, 204, 204); border-left-width: 1px; border-left-style: solid;"><div><div>> > Hi,<br>> > Am 28.10.14 um 12:57 schrieb Krzysztof Pawłowski:<br> <div><div>> > > Hi,<br>> > > Is there any guide about tuning spacewalk performance ? With every new<br>> > > host spacewalk is getting slowly. Using SSM with more than 200-300 hosts<br>> > > is impossible due timeouts. It's also not possible to deploy config<br>> > > files to all hosts.<br>> > > Standard java tuning was done, java gc is not the problem now.<br>> > ><br>> > > Any suggestions ?<br> <br> </div></div>> > what server hardware do you use? What is the systemload while performing<br>> > that tasks? CPU, RAM, Disksystem, IO, Network .... ?<br> <br> <br>> > /Götz<br> <br>> > --<br>> > Götz Reinicke<br>> > IT-Koordinator<br> <br>> > Tel. <a href="tel:%2B49%207141%20969%2082%20420" _djrealurl="tel:%2B49%207141%20969%2082%20420" target="_blank" value="+49714196982420">+49 7141 969 82 420</a><br>> > E-Mail <a href="mailto:goetz.reinicke@filmakademie.de" _djrealurl="mailto:goetz.reinicke@filmakademie.de" target="_blank">goetz.reinicke@filmakademie.de</a><br> <br>> > Filmakademie Baden-Württemberg GmbH<br>> > Akademiehof 10<br>> > 71638 Ludwigsburg<br> <a href="http://www.filmakademie.de" _djrealurl="http://www.filmakademie.de" target="_blank">> > www.filmakademie.de</a><br> <br>> > Eintragung Amtsgericht Stuttgart HRB 205016<br> <br>> > Vorsitzender des Aufsichtsrats: Jürgen Walter MdL<br></div></div></blockquote></div></div></blockquote></div></div></div></div></blockquote></div></div></div></div></blockquote></div></div></div></div></div><br><div>> [The entire original message is not included.]</div> </div> </div> </div>