<div dir="ltr">Nick<div><br></div><div>Recommend you open a new BZ about it and continue with your tests, maybe the developer team will fix asap.</div><div><br></div><div>B'Regards and congratz!</div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature"><div>______________<br>Atenciosamente<br>Waldirio<br>msn: <a href="mailto:waldirio@gmail.com" target="_blank">waldirio@gmail.com</a><br>Skype: waldirio<br>Site: <a href="http://www.waldirio.com.br" target="_blank">www.waldirio.com.br</a><br>Blog: <a href="http://blog.waldirio.com.br" target="_blank">blog.waldirio.com.br</a></div>
<div>LinkedIn: <a href="http://br.linkedin.com/pub/waldirio-pinheiro/22/b21/646" target="_blank">http://br.linkedin.com/pub/waldirio-pinheiro/22/b21/646</a><br>PGP: <a href="http://www.waldirio.com.br/public.html" target="_blank">www.waldirio.com.br/public.html</a></div></div></div>
<br><div class="gmail_quote">On Tue, Dec 16, 2014 at 4:30 PM, Nick Tailor <span dir="ltr"><<a href="mailto:HTailor@bclc.com" target="_blank">HTailor@bclc.com</a>></span> wrote:<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">





<div lang="EN-CA" link="blue" vlink="purple">
<div>
<p style="margin-right:0cm;margin-bottom:12.0pt;margin-left:0cm;line-height:13.35pt;background:white;vertical-align:baseline">
<span style="font-size:10.5pt;font-family:"Arial","sans-serif";color:black">The issue appears to be with oracle a quick restart of it solved the issue.<u></u><u></u></span></p>
<p style="margin-right:0cm;margin-bottom:12.0pt;margin-left:0cm;line-height:13.35pt;background:white;vertical-align:baseline">
<span style="font-size:10.5pt;font-family:"Arial","sans-serif";color:black">So the rhnpush is opening too many concurrent connections which are not closing on the oracle side, so after it hits a certain number it stops listening, similar to that of apache.<u></u><u></u></span></p>
<p style="margin-right:0cm;margin-bottom:12.0pt;margin-left:0cm;line-height:13.35pt;background:white;vertical-align:baseline">
<span style="font-size:10.5pt;font-family:"Arial","sans-serif";color:black">So will need to either increase the concurrent connection or ensure the rhnpush closes the connections after each push.<u></u><u></u></span></p>
<p style="margin-right:0cm;margin-bottom:12.0pt;margin-left:0cm;line-height:13.35pt;background:white;vertical-align:baseline">
<span style="font-size:10.5pt;font-family:"Arial","sans-serif";color:black">Cheers<u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Arial","sans-serif";color:#76b900">Nick Tailor<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#414b56">Nicktailor.com</span><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#76b900"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d"><u></u> <u></u></span></p>
</div>
<p class="MsoNormal"><span style="color:#1f497d"><u></u> <u></u></span></p>
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Nick Tailor
<br>
<b>Sent:</b> December-16-14 10:27 AM<br>
<b>To:</b> <a href="mailto:spacewalk-list@redhat.com" target="_blank">spacewalk-list@redhat.com</a><br>
<b>Subject:</b> RE: rhnpush intermittent 500 internal error<u></u><u></u></span></p>
</div>
</div><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><span style="color:#1f497d">The issue appears to be with oracle.
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d"><u></u> <u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Arial","sans-serif";color:#76b900">Nick Tailor<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#414b56">Nicktalor.com</span><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#76b900"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:#1f497d"><u></u> <u></u></span></p>
</div>
<p class="MsoNormal"><span style="color:#1f497d"><u></u> <u></u></span></p>
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Nick Tailor
<br>
<b>Sent:</b> December-16-14 10:11 AM<br>
<b>To:</b> <a href="mailto:spacewalk-list@redhat.com" target="_blank">spacewalk-list@redhat.com</a><br>
<b>Subject:</b> rhnpush intermittent 500 internal error<u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Has anyone else seen this?<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">I found someone else who had the exact same problem, however no one seemed to have replied to its resolution?<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><a href="http://www.redhat.com/archives/spacewalk-list/2013-February/msg00068.html" target="_blank">http://www.redhat.com/archives/spacewalk-list/2013-February/msg00068.html</a> (Similar problem)<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Has anyone experienced a problem during the rhnpush <u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black;background:white">Internal server error 500 Internal Server Error</span><span style="font-size:13.5pt;color:black"><br>
</span><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black;background:white">Error pushing /root/BAC/TIVsm-JBB.x86_64.rpm:  (500)</span><span style="font-size:13.5pt;color:black"><br>
</span><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black;background:white">Waiting 2 seconds and trying again...</span><span style="font-size:13.5pt;color:black"><br>
</span><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black;background:white">Uploading package /root/BAC/TIVsm-JBB.x86_64.rpm</span><span style="font-size:13.5pt;color:black"><br>
</span><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black;background:white">Using POST request</span><span style="font-size:13.5pt;color:black"><br>
</span><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black;background:white">Internal server error 500 Internal Server Error</span><span style="font-size:13.5pt;color:black"><br>
</span><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black;background:white">Error pushing /root/BAC/TIVsm-JBB.x86_64.rpm:  (500)</span><span style="font-size:13.5pt;color:black"><br>
</span><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black;background:white">Waiting 1 seconds and trying again...</span><span style="font-size:13.5pt;color:black"><br>
</span><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black;background:white">Giving up after 3 attempts<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black;background:white"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black;background:white">I checked the /var/log/rhn/rhn_upload_package_push.log<u></u><u></u></span></b></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black;background:white"><u></u> <u></u></span></b></p>
<p class="MsoNormal"><b>So then I check the oracle tns listener db logs, it appears the listener intermittently for spacewalk stops listening which causes the rhnpush to fail, and return a internal 500 error. I have updated the hosts file and changed the rhn.conf
 to use the ip instead of the db host name to see if that help? I would like to know if anyone else has experienced this and what they did to resolute the issue?<u></u><u></u></b></p>
<p class="MsoNormal"><b><u></u> <u></u></b></p>
<p class="MsoNormal"><b>Please and tahnk you</b><u></u><u></u></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black;background:white"><u></u> <u></u></span></b></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">2014/12/15 20:38:22 -07:00 6407 ::1: server/apacheUploadServer.handler('GET method received, returning',)<u></u><u></u></p>
<p class="MsoNormal">2014/12/15 20:38:22 -07:00 6394 ::1: rhnSQL/driver_cx_Oracle.connect('Connecting to database', 'spacewalk@//<a href="http://kam100.db.com/XE" target="_blank">kam100.db.com/XE</a>')<u></u><u></u></p>
<p class="MsoNormal">2014/12/15 20:38:22 -07:00 6394 ::1: rhnSQL/driver_cx_Oracle.connect('ERROR', 'Connection attempt failed', 12516, 'ORA-12516: TNS:listener could not find available handler with matching protocol stack\n')<u></u><u></u></p>
<p class="MsoNormal">2014/12/15 20:38:22 -07:00 6394 ::1: server/apacheUploadServer._wrapper('ERROR', 'Unhandled exception')<u></u><u></u></p>
<p class="MsoNormal">2014/12/15 20:38:26 -07:00 5510 ::1: package_push/package_push.headerParserHandler('Username', None, 'Force', 0)<u></u><u></u></p>
<p class="MsoNormal"><b><u></u> <u></u></b></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">/listener/alert/log.xml<u></u><u></u></p>
<p class="MsoNormal"><msg time='2014-12-16T09:48:07.721-08:00' org_id='oracle' comp_id='tnslsnr'<u></u><u></u></p>
<p class="MsoNormal">type='UNKNOWN' level='16' host_id= <a href="http://kam100.db.com" target="_blank">kam100.db.com</a><u></u><u></u></p>
<p class="MsoNormal">host_addr='10.0.9.107'><u></u><u></u></p>
<p class="MsoNormal"><txt>16-DEC-2014 09:48:07 * (CONNECT_DATA=(SERVICE_NAME=XE)(CID=(PROGRAM=java)(HOST= <a href="http://kam100.db.com" target="_blank">kam100.db.com</a>)(USER=tomcat))) * (ADDRESS=(PROTOCOL=tcp)(HOST=10.0.9.101)(PORT=45479)) * establish * XE * 12516<u></u><u></u></p>
<p class="MsoNormal"></txt><u></u><u></u></p>
<p class="MsoNormal"></msg><u></u><u></u></p>
<p class="MsoNormal"><msg time='2014-12-16T09:48:07.721-08:00' org_id='oracle' comp_id='tnslsnr'<u></u><u></u></p>
<p class="MsoNormal">type='UNKNOWN' level='16' host_id= <a href="http://kam100.db.com" target="_blank">kam100.db.com</a> host_addr='10.0.9.107'><u></u><u></u></p>
<p class="MsoNormal"><txt>TNS-12516: TNS:listener could not find available handler with matching protocol stack<u></u><u></u></p>
<p class="MsoNormal"></txt><u></u><u></u></p>
<p class="MsoNormal"></msg><u></u><u></u></p>
<p class="MsoNormal"><msg time='2014-12-16T09:48:07.722-08:00' org_id='oracle' comp_id='tnslsnr'<u></u><u></u></p>
<p class="MsoNormal">type='UNKNOWN' level='16' host_id= <a href="http://kam100.db.com" target="_blank">kam100.db.com</a><u></u><u></u></p>
<p class="MsoNormal">host_addr='10.0.9.107'><u></u><u></u></p>
<p class="MsoNormal"><txt>16-DEC-2014 09:48:07 * (CONNECT_DATA=(SERVICE_NAME=XE)(CID=(PROGRAM=java)(HOST= <a href="http://kam100.db.com" target="_blank">kam100.db.com</a>)(USER=tomcat))) * (ADDRESS=(PROTOCOL=tcp)(HOST=10.0.9.101)(PORT=45480)) * establish * XE * 12516<u></u><u></u></p>
<p class="MsoNormal"></txt><u></u><u></u></p>
<p class="MsoNormal"></msg><u></u><u></u></p>
<p class="MsoNormal"><msg time='2014-12-16T09:48:07.722-08:00' org_id='oracle' comp_id='tnslsnr'<u></u><u></u></p>
<p class="MsoNormal">type='UNKNOWN' level='16' host_id= <a href="http://kam100.db.com" target="_blank">kam100.db.com</a><u></u><u></u></p>
<p class="MsoNormal">host_addr='10.0.9.107'><u></u><u></u></p>
<p class="MsoNormal"><txt>TNS-12516: TNS:listener could not find available handler with matching protocol stack<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Arial","sans-serif";color:#76b900">Nick Tailor<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#414b56">Nicktailor.com</span><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#76b900"><u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div></div></div><div><div class="h5">
<hr>
<font face="Arial">This email is intended only for the addressee. It may contain confidential or proprietary information that cannot be disclosed without BCLC's permission. If you have received this email in error, please notify the sender immediately
 and delete the email.</font>
</div></div></div>

<br>_______________________________________________<br>
Spacewalk-list mailing list<br>
<a href="mailto:Spacewalk-list@redhat.com">Spacewalk-list@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/spacewalk-list" target="_blank">https://www.redhat.com/mailman/listinfo/spacewalk-list</a><br></blockquote></div></div>