[Spacewalk-list] osad not picking up actions

[Dimitri Yioulos]

-----Original Message-----
From: spacewalk-list-bounces at redhat.com [mailto:spacewalk-list-bounces at redhat.com] On Behalf Of Nicki Clear
Sent: Thursday, May 19, 2016 4:01 PM
To: spacewalk-list at redhat.com
Subject: [Spacewalk-list] osad not picking up actions


ok -  I know that there have been a lot of messages about osad status and it not working lately, but all of my troubleshooting steps that usually work are not working and I am at a complete loss. I am running spacewalk v
2.4 on centos 6.7

on the webui - my nodes say osad status offline - and only 10 out of 70 nodes are picking up actions.  those 10 say status is online

When I run
(I replaced  the actual fqdn of spacewalk with <spacewalk fqdn> )

osad -N -vvvv


2016-05-19 14:39:35 jabber_lib.subscribe_to_presence: Subscribed to {'rhn-dispatcher-sat@<spacewalk fqdn>': {'jid':
'rhn-dispatcher-sat@<spacewalk fqdn>', 'subscription': u'to'}}
2016-05-19 14:39:35 jabber_lib.subscribe_to_presence: Subscribed both {}
2016-05-19 14:39:35 jabber_lib.subscribe_to_presence: Subscribed none {}
2016-05-19 14:39:35 jabber_lib.subscribe_to_presence: Subscribed from {}
2016-05-19 14:39:35 jabber_lib.subscribe_to_presence: Subscribed to
2016-05-19 14:39:35 jabber_lib.send_presence: None None
2016-05-19 14:39:35 jabber_lib.process_forever:
2016-05-19 14:39:35 jabber_lib.process: 180

eventually it will fail with

2016-05-19 15:54:54 jabber_lib.process: 180 Error caught:
Traceback (most recent call last):
  File "/usr/share/rhn/osad/jabber_lib.py", line 120, in main
    self.process_forever(c)
  File "/usr/share/rhn/osad/jabber_lib.py", line 178, in process_forever
    self.process_once(client)
  File "/usr/share/rhn/osad/osad.py", line 249, in process_once
    client.process(timeout=180)
  File "/usr/share/rhn/osad/jabber_lib.py", line 1040, in process
    data = self._read(self.BLOCK_SIZE)
SSLError: ('OpenSSL error; will retry', "(-1, 'Unexpected EOF')")

2016-05-19 15:54:54 jabber_lib.main: Sleeping 115 seconds


so it looks like it is subscribed to the spacewalk server,  but it still doesnt work.

to troubleshoot -

on the spacewalk server I ran

spacewalk-service stop
rm -rf /var/lib/jabberd/db/*
spacewalk-service start


and on the clients I ran

rhn-profile-sync
rhn-actions-control --enable-all
rm -rf /etc/sysconfig/rhn/osad-auth.conf /etc/init.d/osad restart

and then ran
osad -N -vvvv

to check if it connects or picks up the action... which it doesnt

There arent a whole lot of messages in /var/log/rhn/osa-dispatcher.log

2016/05/19 14:32:45 -04:00 9177 0.0.0.0: osad/jabber_lib.__init__
2016/05/19 14:32:45 -04:00 9177 0.0.0.0: osad/jabber_lib.setup_connection ('Connected to jabber server', '<spacewalk fqdn>')
2016/05/19 14:32:45 -04:00 9177 0.0.0.0: osad/osa_dispatcher.fix_connection
('Upstream notification server started on port', 1290)
2016/05/19 14:32:45 -04:00 9177 0.0.0.0: osad/jabber_lib.process_forever
2016/05/19 14:37:03 -04:00 9177 0.0.0.0:
osad/dispatcher_client._message_callback('Ping response',)
2016/05/19 14:42:29 -04:00 10363 0.0.0.0: osad/jabber_lib.__init__
2016/05/19 14:42:29 -04:00 10363 0.0.0.0: osad/jabber_lib.setup_connection ('Connected to jabber server', '<spacewalk fqdn>')
2016/05/19 14:42:29 -04:00 10363 0.0.0.0:
osad/osa_dispatcher.fix_connection('Upstream notification server started on port', 1290)
2016/05/19 14:42:30 -04:00 10363 0.0.0.0: osad/jabber_lib.process_forever
2016/05/19 14:48:28 -04:00 10363 0.0.0.0:
osad/dispatcher_client._message_callback('Ping response',)


Does anyone have any ideas?

Thanks!


Hi, Nikki.

What version of yum-rhn-plugin is running on your clients?  And, are they all running Centos 6.x?

Dimitri