[Spacewalk-list] Spacewalk 1.7 Postgres out of connections

[Brown, Rodrick]

This might seem like a stretch but another alternative may be to moving to the Oracle-XE version which can support a lot more convections than postgres.

From: spacewalk-list-bounces at redhat.com [mailto:spacewalk-list-bounces at redhat.com] On Behalf Of Paul Robert Marino
Sent: Friday, May 11, 2012 6:56 PM
To: spacewalk-list at redhat.com
Subject: Re: [Spacewalk-list] Spacewalk 1.7 Postgres out of connections


Well I've worked around it by reducing the tcp heartbeat to 10 minutes in /etc/sysctl.conf and reducing the idle client timeout in postgresqls configuration.
But yes there is an issue with this.
I think it the monitoring that does it but I'm not sure. I can tell you the incidents of it do go up with monitoring turned on.
The problem is is seems like its a shared library that isn't logging if the class goes out of scope out of the database correctly that make tracking the root cause significantly harder.
I'm not familiar enough with python to figure explain how it should be fixed but in perl the DBI driver had. DESTROY method that catches if the programer forgot to logout of the database before the instance of the class goes out of scope or the application exits. It seems to me this is missing from the python postgresql driver and if that functionality could be added it would resolve the issue.
On May 11, 2012 6:13 PM, "Scott Piazza" <scott.piazza at bespokess.com<mailto:scott.piazza at bespokess.com>> wrote:
I recently upgraded from Spacewalk 1.6 to 1.7.  In this version, I am
seeing an issue where Spacewalk works fine for several hours, but then
the database runs out of connections, and Spacewalk begins throwing
errors.  Similar Postgresql errors on other forums try to blame the
application for not closing connections correctly.  Older Spacewalk
posts online seem to suggest there's an issue with osa-dispatcher that
might be causing it.  I have disabled osa-dispatcher for now and am
waiting to see if that helps narrow down the source of the problem.
However, if anyone has additional suggestions to get this fixed, I would
appreciate hearing them.


Server: CentOS 6.2
Spacwalk: 1.7
SELinux: off
PostgreSQL: 8.4.9-1.el6_1.1
Postgres max connections: 500

/var/log/rhn/rhn_server_xmlrpc.log:
2012/05/10 12:17:50 -07:00 29438 10.0.11.2<http://10.0.11.2>:
rhnSQL/driver_postgresql.check_connection('ERROR', "DATABASE CONNECTION
TO 'whmgmt' LOST", "Exception information: Database instance has no
attribute 'dbh'")

/var/log/rhn/osa-dispatcher.log:
2012/05/10 08:14:09 -07:00 10255 0.0.0.0<http://0.0.0.0>:
rhnSQL/driver_postgresql.check_connection('ERROR', "DATABASE CONNECTION
TO 'whmgmt' LOST", 'Exception information: server closed the connection
unexpectedly\n\tThis probably means the server terminated abnormally\n
\tbefore or while processing the request.\n')

/var/lib/pgsql/data/pg_log/logfile.log:
2012-05-10 17:50:47 MST - whmgmtuser - [local]: FATAL:  sorry, too many
clients already
2012-05-10 17:50:49 MST - whmgmtuser - 127.0.0.1<http://127.0.0.1>: LOG:  unexpected EOF
on client connection
2012-05-10 17:51:13 MST - whmgmtuser - [local]: LOG:  could not send
data to client: Broken pipe


_______________________________________________
Spacewalk-list mailing list
Spacewalk-list at redhat.com<mailto:Spacewalk-list at redhat.com>
https://www.redhat.com/mailman/listinfo/spacewalk-list

Please visit our website for important disclaimers/disclosures regarding Knight's products and services:

http://knight.com/KnightEmailDisclaimer.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20120512/3d39655d/attachment.htm>