[Spacewalk-list] rhnpush intermittent 500 internal error

[Nick Tailor]

The issue appears to be with oracle a quick restart of it solved the issue.

So the rhnpush is opening too many concurrent connections which are not closing on the oracle side, so after it hits a certain number it stops listening, similar to that of apache.

So will need to either increase the concurrent connection or ensure the rhnpush closes the connections after each push.

Cheers
Nick Tailor
Nicktailor.com


From: Nick Tailor
Sent: December-16-14 10:27 AM
To: spacewalk-list at redhat.com
Subject: RE: rhnpush intermittent 500 internal error

The issue appears to be with oracle.


Nick Tailor
Nicktalor.com


From: Nick Tailor
Sent: December-16-14 10:11 AM
To: spacewalk-list at redhat.com<mailto:spacewalk-list at redhat.com>
Subject: rhnpush intermittent 500 internal error

Has anyone else seen this?

I found someone else who had the exact same problem, however no one seemed to have replied to its resolution?

http://www.redhat.com/archives/spacewalk-list/2013-February/msg00068.html (Similar problem)

Has anyone experienced a problem during the rhnpush

Internal server error 500 Internal Server Error
Error pushing /root/BAC/TIVsm-JBB.x86_64.rpm:  (500)
Waiting 2 seconds and trying again...
Uploading package /root/BAC/TIVsm-JBB.x86_64.rpm
Using POST request
Internal server error 500 Internal Server Error
Error pushing /root/BAC/TIVsm-JBB.x86_64.rpm:  (500)
Waiting 1 seconds and trying again...
Giving up after 3 attempts

I checked the /var/log/rhn/rhn_upload_package_push.log

So then I check the oracle tns listener db logs, it appears the listener intermittently for spacewalk stops listening which causes the rhnpush to fail, and return a internal 500 error. I have updated the hosts file and changed the rhn.conf to use the ip instead of the db host name to see if that help? I would like to know if anyone else has experienced this and what they did to resolute the issue?

Please and tahnk you


2014/12/15 20:38:22 -07:00 6407 ::1: server/apacheUploadServer.handler('GET method received, returning',)
2014/12/15 20:38:22 -07:00 6394 ::1: rhnSQL/driver_cx_Oracle.connect('Connecting to database', 'spacewalk@//kam100.db.com/XE')
2014/12/15 20:38:22 -07:00 6394 ::1: rhnSQL/driver_cx_Oracle.connect('ERROR', 'Connection attempt failed', 12516, 'ORA-12516: TNS:listener could not find available handler with matching protocol stack\n')
2014/12/15 20:38:22 -07:00 6394 ::1: server/apacheUploadServer._wrapper('ERROR', 'Unhandled exception')
2014/12/15 20:38:26 -07:00 5510 ::1: package_push/package_push.headerParserHandler('Username', None, 'Force', 0)


/listener/alert/log.xml
<msg time='2014-12-16T09:48:07.721-08:00' org_id='oracle' comp_id='tnslsnr'
type='UNKNOWN' level='16' host_id= kam100.db.com
host_addr='10.0.9.107'>
<txt>16-DEC-2014 09:48:07 * (CONNECT_DATA=(SERVICE_NAME=XE)(CID=(PROGRAM=java)(HOST= kam100.db.com)(USER=tomcat))) * (ADDRESS=(PROTOCOL=tcp)(HOST=10.0.9.101)(PORT=45479)) * establish * XE * 12516
</txt>
</msg>
<msg time='2014-12-16T09:48:07.721-08:00' org_id='oracle' comp_id='tnslsnr'
type='UNKNOWN' level='16' host_id= kam100.db.com host_addr='10.0.9.107'>
<txt>TNS-12516: TNS:listener could not find available handler with matching protocol stack
</txt>
</msg>
<msg time='2014-12-16T09:48:07.722-08:00' org_id='oracle' comp_id='tnslsnr'
type='UNKNOWN' level='16' host_id= kam100.db.com
host_addr='10.0.9.107'>
<txt>16-DEC-2014 09:48:07 * (CONNECT_DATA=(SERVICE_NAME=XE)(CID=(PROGRAM=java)(HOST= kam100.db.com)(USER=tomcat))) * (ADDRESS=(PROTOCOL=tcp)(HOST=10.0.9.101)(PORT=45480)) * establish * XE * 12516
</txt>
</msg>
<msg time='2014-12-16T09:48:07.722-08:00' org_id='oracle' comp_id='tnslsnr'
type='UNKNOWN' level='16' host_id= kam100.db.com
host_addr='10.0.9.107'>
<txt>TNS-12516: TNS:listener could not find available handler with matching protocol stack



Nick Tailor
Nicktailor.com


________________________________
This email is intended only for the addressee. It may contain confidential or proprietary information that cannot be disclosed without BCLC's permission. If you have received this email in error, please notify the sender immediately and delete the email.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20141216/2b805989/attachment.htm>