Well most new install should be using PostgreSQL it gets significantly better performance than Oracle also most people turn off osad once they get past 400 managed nodes and just rely on rhnsd.<br><br>Further more with that many host you are definitely better off using an external database server. Preferably PostgreSQL 9.2 because the replication options are better but you must ensure that the right JDBC drivers are properly installed on the spacewalk servers or you will have issues with Taskomatic.<br><br><span style="font-family:Prelude, Verdana, san-serif;"><br><br></span><span id="signature"><div style="font-family: arial, sans-serif; font-size: 12px;color: #999999;">-- Sent from my HP Pre3</div><br></span><span style="color:navy; font-family:Prelude, Verdana, san-serif; "><hr align="left" style="width:75%">On Apr 18, 2014 17:48, Matthew Madey <mattmadey@gmail.com> wrote: <br><br></span><div dir="ltr">Looking to get some feedback on performance settings any of you may have done on your Spacewalk server and\or external Oracle Database servers.<div><br></div><div>I'm looking to patch 1600 systems simultaneously to meet our business requirements.. at our current settings with an 8 CPU \ 32 GB RAM Spacewalk server, I can patch 400 systems simultaneously without much issue. Much of our bottleneck appears to be disk await time on the Oracle database side, which we are addressing, but I'm wondering if there are other kernel\tomcat\apache settings I should be looking at to improve handling of all the XMLRPC traffic coming in to the Spacewalk server. </div>
<div>400 systems appears to be our hard limit at the moment, adding just an additional 50 or so leads to some pretty catastrophic effects. I haven't seen any GC or JVM errors in catalina.out thus far, but our memory usage goes through the roof and begin swapping badly. My guess is that the wait time on the database is so bad the Spacewalk server starts backlogging requests until it runs out of memory. </div>
<div><br></div><div>Has anyone else attempted to patch this volume of systems at one time? If so, what changes did you make to the server configuration to accommodate that? </div><div><br></div><div><br></div><div><br></div>
<div>Below are some of the network tuning parameters we've entered. </div><div><br></div><div><div>net.ipv4.tcp_max_syn_backlog = 8192</div><div>net.ipv4.tcp_syncookies = 1</div><div>net.ipv4.conf.all.rp_filter = 1</div>
<div>net.ipv4.conf.all.accept_source_route = 0</div><div>net.ipv4.conf.all.accept_redirects = 0</div><div>net.ipv4.conf.all.secure_redirects = 0</div><div>net.ipv4.conf.default.rp_filter = 1</div><div>net.ipv4.conf.default.accept_source_route = 0</div>
<div>net.ipv4.conf.default.accept_redirects = 0</div><div>net.ipv4.conf.default.secure_redirects = 0</div><div>net.ipv4.icmp_echo_ignore_broadcasts = 1</div><div>net.ipv4.ip_forward = 0</div><div>net.ipv4.conf.all.send_redirects = 0</div>
<div>net.ipv4.conf.default.send_redirects = 0</div><div>net.ipv4.neigh.default.unres_qlen = 3</div><div><br></div><div>#Added OSE Tuning</div><div>###8x normal for faster network queue draining</div><div>net.core.dev_weight = 512</div>
<div>#</div><div>###3x normal for a queue and budget suited to networks greater than 100mbps</div><div>net.core.netdev_budget = 10000</div><div>net.core.netdev_max_backlog = 30000</div><div>net.core.rmem_max = 4194304 </div>
<div>net.core.wmem_max = 1048586</div><div><br></div><div>####min, default (BDP), max (2xBDP) for window scaling (BDP calculated for 10gbps link speed and near zero latency in a LAN)</div><div>net.ipv4.tcp_rmem = 4096 87380 4194304</div>
<div>net.ipv4.tcp_wmem = 4096 16384<span class="" style="white-space:pre"> </span>4194304</div><div><br></div><div>####Affects UDP streams similarly to the above for TCP.</div><div>net.core.rmem_default = 312500</div><div>
net.core.wmem_default = 312500</div><div><br></div><div>###Enable Packetization Layer Path MTU Discovery (PLPMTUD)</div><div>net.ipv4.tcp_mtu_probing = 1</div><div><br></div><div>net.core.somaxconn = 1536</div></div><div>
<br></div></div>