[Spacewalk-list] Spacewalk 1.7 w/ postgresql crashed

[Paul Robert Marino]

Postgresql was killed with a -9 which means some one hard killed the
process then restarted it. Looks like some one was doing something naughty
on your box.
This is not a spacewalk problem this is a sysadmin who made a mistake then
didn't fess to it.
On Nov 19, 2012 4:18 AM, "Wolfgang Neudorfer" <mlist at woifi.at> wrote:

> Hi,
>
> starting Saturday 17/11/2012 01:46, our Spacewalk server started to send
> out multiple mails per minute (probably on each connection attempt of a
> client?) like this:
>
> -------------------------------------------------
> RHN TRACEBACK from spacewalk1:
>
> Exception reported from spacewalk1
> Time: Sat Nov 17 01:45:30 2012
> Exception type <class 'spacewalk.server.rhnSQL.sql_base.SQLConnectError'>
> Request object information:
> URI: /XMLRPC
> Remote Host: 192.168.254.xxx
> Server Name: spacewalk1:443
> Headers passed in:
>     Accept-Encoding: identity
>     CONTENT_LENGTH: 2325
>     CONTENT_TYPE: text/xml
>     DOCUMENT_ROOT: /var/www/html
>     GATEWAY_INTERFACE: CGI/1.1
>     HTTPS: 1
>     HTTP_ACCEPT_ENCODING: identity
>     HTTP_HOST: spacewalk1
>     HTTP_USER_AGENT: rhn.rpclib.py/$Revision$
>     HTTP_X_CLIENT_VERSION: 1
>     HTTP_X_INFO: RPC Processor (C) Red Hat, Inc (version $Revision$)
>     HTTP_X_RHN_TRANSPORT_CAPABILITY: follow-redirects=3
>     HTTP_X_TRANSPORT_INFO: Extended Capabilities Transport (C) Red Hat,
> Inc (version $Revision$)
>     Host: tsasecspacewalk1.sec
>     PATH_INFO:
>     QUERY_STRING:
>     REMOTE_ADDR: 192.168.254.xxx
>     REMOTE_PORT: 59649
>     REQUEST_METHOD: POST
>     REQUEST_URI: /XMLRPC
>     SCRIPT_FILENAME: /usr/share/rhn/wsgi/xmlrpc.py
>     SCRIPT_NAME: /XMLRPC
>     SCRIPT_URI: https://tsasecspacewalk1.sec/XMLRPC
>     SCRIPT_URL: /XMLRPC
>     SERVER_ADDR: 192.168.254.xxx
>     SERVER_ADMIN: root at localhost
>     SERVER_NAME: spacewalk1
>     SERVER_PORT: 443
>     SERVER_PROTOCOL: HTTP/1.1
>     SERVER_SIGNATURE: <address>Apache Server at spacewalk1 Port
> 443</address>
>
>     SERVER_SOFTWARE: Apache
>     User-Agent: rhn.rpclib.py/$Revision$
>     X-Client-Version: 1
>     X-Info: RPC Processor (C) Red Hat, Inc (version $Revision$)
>     X-RHN-Transport-Capability: follow-redirects=3
>     X-Transport-Info: Extended Capabilities Transport (C) Red Hat, Inc
> (version $Revision$)
>     mod_wsgi.application_group: tsasecspacewalk1.sec|/xmlrpc
>     mod_wsgi.callable_object: application
>     mod_wsgi.handler_script:
>     mod_wsgi.input_chunked: 0
>     mod_wsgi.listener_host:
>     mod_wsgi.listener_port: 443
>     mod_wsgi.process_group:
>     mod_wsgi.request_handler: wsgi-script
>     mod_wsgi.script_reloading: 1
>     mod_wsgi.version: (3, 2)
>     wsgi.errors: <mod_wsgi.Log object at 0x7f8e4a83d370>
>     wsgi.file_wrapper: <built-in method file_wrapper of mod_wsgi.Adapter
> object at 0x7f8e4a83c300>
>     wsgi.input: <mod_wsgi.Input object at 0x7f8e4a83d330>
>     wsgi.multiprocess: True
>     wsgi.multithread: False
>     wsgi.run_once: False
>     wsgi.url_scheme: https
>     wsgi.version: (1, 1)
> -------------------------------------------------
>
> Apparently, something happend to the postgres server. In the log I see:
>
> -------------------------------------------------
> LOG:  server process (PID 31999) was terminated by signal 9: Killed
> LOG:  terminating any other active server processes
> WARNING:  terminating connection because of crash of another server process
> DETAIL:  The postmaster has commanded this server process to roll back the
> current transaction and exit, because another server process exited
> abnormally and possibly corrupted shared memory.
>
> ... (the last 2 lines appear multiple times)
>
> FATAL:  the database system is in recovery mode
> FATAL:  the database system is in recovery mode
> FATAL:  the database system is in recovery mode
> FATAL:  the database system is in recovery mode
>
> ... (this line apprears multiple times)
> -------------------------------------------------
>
> The harddisk was not full, also RAM was ok. I restarted the host and
> Spacewalk seems to be fine. I can login an all hosts are there.
>
> Any hints? I am running Spacewalk 1.7 on CentOS x64 6.3 with PostgresSQL
> 8.4.13.
>
> Thanks,
>
> Wolfgang
>
> _______________________________________________
> Spacewalk-list mailing list
> Spacewalk-list at redhat.com
> https://www.redhat.com/mailman/listinfo/spacewalk-list
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20121119/54a8f16d/attachment.htm>