[Spacewalk-list] Spacewalk 2.2: Clients are not picking up scheduled tasks

[Brian Musson]

Hi I am trying to get to the bottom of this issue but I have hit a wall.
Any hints would be greatly appreciated. The action gets scheduled but never
gets picked up. From last week the jobs are still "Queued" and have not
been picked up yet. I have a task to do this same patching to a larger pool
of systems (almost 800) this week. Any direction or hints would be greatly
appreciated.


This action will be executed after 6/11/15 6:00:00 AM PDT
This action's status is: Queued.
This action has not yet been picked up.

BM

On Fri, Jun 12, 2015 at 1:09 PM, Brian Musson <mrbrian at gmail.com> wrote:

> Sorry to double post, but i thought it may be useful to show the output of
> the spacewalk proxy's /var/log/rhn/rhn_proxy_broker.log at the time of the
> run.
>
> 10.12.82.141 = client
>
> spacewalk-proxy# tail -f /var/log/rhn/rhn_proxy_broker.log
>
> 2015/06/12 16:06:26 -04:00 11778 10.12.82.41:
> proxy/apacheServer.__call__('New request, component proxy.broker',)
> 2015/06/12 16:06:26 -04:00 11778 10.12.82.141: broker/rhnBroker.handler
> 2015/06/12 16:06:26 -04:00 11778 10.12.82.141:
> proxy/rhnShared._serverCommo
> 2015/06/12 16:06:26 -04:00 11778 10.12.82.141:
> broker/rhnBroker.__handleAction
> 2015/06/12 16:06:26 -04:00 11778 10.12.82.141:
> proxy/rhnShared._clientCommo
> 2015/06/12 16:06:26 -04:00 11778 10.12.82.141:
> proxy/rhnShared._forwardServer2Client
> 2015/06/12 16:06:26 -04:00 11778 10.12.82.141:
> proxy/apacheHandler.handler('Leaving with status code 0',)
> 2015/06/12 16:06:26 -04:00 11778 10.12.82.141:
> proxy/apacheHandler.cleanupHandler
> 2015/06/12 16:06:26 -04:00 11781 192.168.1.208:
> proxy/apacheServer.__call__('New request, component proxy.broker',)
> 2015/06/12 16:06:26 -04:00 11781 10.12.82.141: broker/rhnBroker.handler
> 2015/06/12 16:06:26 -04:00 11781 10.12.82.141:
> proxy/rhnShared._serverCommo
> 2015/06/12 16:06:26 -04:00 11781 10.12.82.141:
> broker/rhnBroker.__handleAction
> 2015/06/12 16:06:26 -04:00 11781 10.12.82.141:
> proxy/rhnShared._clientCommo
> 2015/06/12 16:06:26 -04:00 11781 10.12.82.141:
> proxy/rhnShared._forwardServer2Client
> 2015/06/12 16:06:26 -04:00 11781 10.12.82.141:
> proxy/apacheHandler.handler('Leaving with status code 0',)
> 2015/06/12 16:06:26 -04:00 11781 10.12.82.141:
> proxy/apacheHandler.cleanupHandler
> 2015/06/12 16:06:26 -04:00 11777 10.12.72.56:
> proxy/apacheServer.__call__('New request, component proxy.broker',)
> 2015/06/12 16:06:26 -04:00 11777 10.12.82.141: broker/rhnBroker.handler
> 2015/06/12 16:06:26 -04:00 11777 10.12.82.141:
> proxy/rhnShared._serverCommo
> 2015/06/12 16:06:26 -04:00 11777 10.12.82.141:
> broker/rhnBroker.__handleAction
> 2015/06/12 16:06:26 -04:00 11777 10.12.82.141:
> proxy/rhnShared._clientCommo
> 2015/06/12 16:06:26 -04:00 11777 10.12.82.141:
> proxy/rhnShared._forwardServer2Client
> 2015/06/12 16:06:26 -04:00 11777 10.12.82.141:
> proxy/apacheHandler.handler('Leaving with status code 0',)
> 2015/06/12 16:06:26 -04:00 11777 10.12.82.141:
> proxy/apacheHandler.cleanupHandler
> 2015/06/12 16:06:26 -04:00 11784 10.12.72.41:
> proxy/apacheServer.__call__('New request, component proxy.broker',)
> 2015/06/12 16:06:26 -04:00 11784 10.12.82.141: broker/rhnBroker.handler
> 2015/06/12 16:06:26 -04:00 11784 10.12.82.141:
> proxy/rhnShared._serverCommo
> 2015/06/12 16:06:26 -04:00 11784 10.12.82.141:
> broker/rhnBroker.__handleAction
> 2015/06/12 16:06:26 -04:00 11784 10.12.82.141:
> proxy/rhnShared._clientCommo
> 2015/06/12 16:06:26 -04:00 11784 10.12.82.141:
> proxy/rhnShared._forwardServer2Client
> 2015/06/12 16:06:26 -04:00 11784 10.12.82.141:
> proxy/apacheHandler.handler('Leaving with status code 0',)
> 2015/06/12 16:06:26 -04:00 11784 10.12.82.141:
> proxy/apacheHandler.cleanupHandler
>
> BM
>
> On Fri, Jun 12, 2015 at 12:37 PM, Brian Musson <mrbrian at gmail.com> wrote:
>
>> My server connects indirectly through a proxy due to network
>> segmentation. Checked the proxy for that file but could not find it. I did
>> a tail on the spacewalk master and saw lots of messages mentioning the
>> proxy server that serves the clients in question:
>>
>> 10.12.13.142 = our proxy
>> 1000010038 = client ID
>>
>> spacewalk-master# tail -f /var/log/rhn/rhn_server_xmlrpc.log | grep
>> 10.12.13.142
>> 2015/06/12 12:34:14 -07:00 16704 10.12.13.142:
>> xmlrpc/queue.get(1000010038, 2, 'checkins enabled')
>> 2015/06/12 12:34:15 -07:00 16718 10.12.13.142:
>> xmlrpc/up2date.listChannels(1000010038,)
>> 2015/06/12 12:34:15 -07:00 16721 10.12.13.142:
>> xmlrpc/registration.welcome_message('lang: None',)
>>
>>
>>
>> BM
>>
>> On Fri, Jun 12, 2015 at 5:04 AM, Jan Dobes <jdobes at redhat.com> wrote:
>>
>>> ----- Original Message -----
>>> > From: "Brian Musson" <mrbrian at gmail.com>
>>> > To: spacewalk-list at redhat.com
>>> > Sent: Thursday, June 11, 2015 8:43:47 PM
>>> > Subject: [Spacewalk-list] Spacewalk 2.2: Clients are not picking up
>>>  scheduled tasks
>>> >
>>> > I have about 3000 systems registered in spacewalk, but today we are
>>> focusing
>>> > on applying package updates to 22 of them. Of the 22 systems scheduled
>>> to
>>> > have security errata applied to them, 20 successfully completed the
>>> update
>>> > without error. Unfortunately, there are two systems which have the task
>>> > queued and have not picked it up yet.
>>> >
>>> > I have restarted osad, rhnsd, restarted jabberd on the spacewalk
>>> master and
>>> > proxy through which these failed systems connect. Other clients which
>>> have
>>> > successfully updated go through this proxy server as well.
>>> >
>>> > When looking at the GUI, the client appears to be healthy.
>>> >
>>> > BM
>>> >
>>>
>>> What will appear in '/var/log/rhn/rhn_server_xmlrpc.log' on the
>>> spacewalk server when you run rhn_check?
>>>
>>> --
>>> Jan Dobes
>>> Satellite Engineering, Red Hat
>>>
>>> _______________________________________________
>>> Spacewalk-list mailing list
>>> Spacewalk-list at redhat.com
>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20150615/124abcef/attachment.htm>