[Spacewalk-list] Clients not picking up requested changes, period

Glen Collins glenc2004 at comcast.net
Tue Jan 27 15:28:00 UTC 2015


Hi Paulo, 

Why don't you try this on the client.... 

service osad stop 
rm -rf /etc/sysconfig/rhn/osad-auth.conf 
service osad start 
(Wait for 2 minutes) and run (On the client) 
/usr/sbin/rhn_check 

I've had the same issues and just restarting OSAD didn't do it for me. But when I removed the auth file, that seemed to fix things. Not elegant in any sort of means but it seems to work. Have not tried it long term since I'm putting this "fix" into place right now. I guess this is what happens when you cobble a product together and "make it work". I have found jabber to be a PITA! 

Anyways, hope this helps! 

Glen Collins 

----- Original Message -----

It's not permissions: 

[root at zbx-prx-elk-04 ~]# rhn-actions-control --report 
deploy is enabled 
diff is enabled 
upload is enabled 
mtime_upload is enabled 
run is enabled 

It's not SSL either, we have our systems configured to speak in the clear while troubleshooting this issue. Flipping over to HTTPS doesn't change anything. 

This particular server has three tasks pending now: two config file pushes and one remote command (just to run "date"). Here's the output of rhn_check -vvvv: 

[root at zbx-prx-elk-04 ~]# rhn_check -vvvv 
D: opening db environment /var/lib/rpm cdb:mpool:joinenv 
D: opening db index /var/lib/rpm/Packages rdonly mode=0x0 
D: locked db index /var/lib/rpm/Packages 
D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key 
D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key 
D: loading keyring from rpmdb 
D: opening db index /var/lib/rpm/Name rdonly mode=0x0 
D: added key gpg-pubkey-b6f8c6bf-48853124 to keyring 
D: added key gpg-pubkey-8db2536f-4ade0b9d to keyring 
D: added key gpg-pubkey-66fd4949-4803fe57 to keyring 
D: added key gpg-pubkey-c105b9de-4e0fd3a3 to keyring 
D: added key gpg-pubkey-217521f6-45e8a532 to keyring 
D: added key gpg-pubkey-0608b895-4bd22942 to keyring 
D: added key gpg-pubkey-863a853d-4f55f54d to keyring 
D: added key gpg-pubkey-79ea5ed4-508d25a6 to keyring 
D: Using legacy gpg-pubkey(s) from rpmdb 
D: opening db index /var/lib/rpm/Providename rdonly mode=0x0 
D: do_call packages.checkNeedUpdate('rhnsd=1',){} 
D: opening db environment /var/lib/rpm cdb:mpool:joinenv 
D: opening db index /var/lib/rpm/Packages rdonly mode=0x0 
D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key 
D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key 
D: loading keyring from rpmdb 
D: opening db index /var/lib/rpm/Name rdonly mode=0x0 
D: added key gpg-pubkey-b6f8c6bf-48853124 to keyring 
D: added key gpg-pubkey-8db2536f-4ade0b9d to keyring 
D: added key gpg-pubkey-66fd4949-4803fe57 to keyring 
D: added key gpg-pubkey-c105b9de-4e0fd3a3 to keyring 
D: added key gpg-pubkey-217521f6-45e8a532 to keyring 
D: added key gpg-pubkey-0608b895-4bd22942 to keyring 
D: added key gpg-pubkey-863a853d-4f55f54d to keyring 
D: added key gpg-pubkey-79ea5ed4-508d25a6 to keyring 
D: Using legacy gpg-pubkey(s) from rpmdb 
D: opening db index /var/lib/rpm/Providename rdonly mode=0x0 
D: closed db index /var/lib/rpm/Providename 
D: closed db index /var/lib/rpm/Name 
D: closed db index /var/lib/rpm/Packages 
D: closed db environment /var/lib/rpm 
Loaded plugins: fastestmirror, rhnplugin 
Config time: 0.040 
D: login(forceUpdate=False) invoked 
D: readCachedLogin invoked 
D: Checking pickled loginInfo, currentTime=1422370609.99, createTime=1422370587.99, expire-offset=3600.0 
D: readCachedLogin(): using pickled loginInfo set to expire at 1422374187.99 
D: rpcServer: Calling XMLRPC up2date.listChannels 
Setting up Package Sacks 
Loading mirror speeds from cached hostfile 
pkgsack time: 0.071 
rpmdb time: 0.000 
Loading mirror speeds from cached hostfile 
repo time: 0.000 
D: local action status: (0, 'rpm database not modified since last update (or package list recently updated)', {}) 
D: rpcServer: Calling XMLRPC registration.welcome_message 
D: closed db index /var/lib/rpm/Providename 
D: closed db index /var/lib/rpm/Name 
D: closed db index /var/lib/rpm/Packages 
D: closed db environment /var/lib/rpm 

Paulo: To answer your question about the API, spacecmd calls it extensively; and you can look up pending tasks by ID. 

spacecmd {SSM:0}> schedule_details 578593 
ID: 578593 
Action: Remote Command on zbx-prx-elk-04.company.com . 
User: admincswingler 
Date: 20150126T16:36:00 

Completed: 0 
Failed: 0 
Pending: 1 

Pending Systems 
--------------- 
zbx-prx-elk-04.company.com 
spacecmd {SSM:0}> schedule_details 578588 
ID: 578588 
Action: Deploy config files to system 
User: admincswingler 
Date: 20150126T16:22:00 

Completed: 0 
Failed: 0 
Pending: 1 

Pending Systems 
--------------- 
zbx-prx-elk-04.company.com 
spacecmd {SSM:0}> schedule_details 578586 
ID: 578586 
Action: Deploy config files to system 
User: admincswingler 
Date: 20150126T16:18:00 

Completed: 0 
Failed: 0 
Pending: 1 

Pending Systems 
--------------- 
zbx-prx-elk-04.company.com 
spacecmd {SSM:0}> 

Still scratching my head on this one, and getting rather frustrated :) 

On Tue, Jan 27, 2015 at 7:02 AM, Paulo Silva < paulojjs at gmail.com > wrote: 



I'm checking pending events using the web interface (URL /rhn/systems/details/history/Pending.do?sid=1000010296) and it shows 2 pending events: and Hardware Refresh and an Errata Update. 

The sid seams to be correct with the server I'm running my commands: 

# grep 1000010296 /etc/sysconfig/rhn/systemid 
<value><string>ID-1000010296</string></value> 

Is there a way to check for pending events using the API? 

2015-01-27 12:45 GMT+00:00 Alexander Innes < senni at necurity.co.uk > : 

<blockquote>

Just to make sure im not thinking of the wrong feture are you trying to use to remote commands? 
Systems -> System -> somethin -> remote command (or a different one so i can test in our environment :) ) 

from that output its not seeing anything as you said :o 

On 27 January 2015 at 12:16, Paulo Silva < paulojjs at gmail.com > wrote: 

<blockquote>

I'm having the same issue, running rhn_check does this: 

# rhn_check -vvv 
D: opening db environment /var/lib/rpm cdb:mpool:joinenv 
D: opening db index /var/lib/rpm/Packages rdonly mode=0x0 
D: locked db index /var/lib/rpm/Packages 
D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key 
D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key 
D: loading keyring from rpmdb 
D: opening db index /var/lib/rpm/Name rdonly mode=0x0 
D: added key gpg-pubkey-c105b9de-4e0fd3a3 to keyring 
D: added key gpg-pubkey-b5c61460-41667588 to keyring 
D: added key gpg-pubkey-6b8d79e6-3f49313d to keyring 
D: added key gpg-pubkey-0608b895-4bd22942 to keyring 
D: added key gpg-pubkey-863a853d-4f55f54d to keyring 
D: Using legacy gpg-pubkey(s) from rpmdb 
D: opening db index /var/lib/rpm/Providename rdonly mode=0x0 
D: do_call packages.checkNeedUpdate('rhnsd=1',){} 
D: opening db environment /var/lib/rpm cdb:mpool:joinenv 
D: opening db index /var/lib/rpm/Packages rdonly mode=0x0 
D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key 
D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key 
D: loading keyring from rpmdb 
D: opening db index /var/lib/rpm/Name rdonly mode=0x0 
D: added key gpg-pubkey-c105b9de-4e0fd3a3 to keyring 
D: added key gpg-pubkey-b5c61460-41667588 to keyring 
D: added key gpg-pubkey-6b8d79e6-3f49313d to keyring 
D: added key gpg-pubkey-0608b895-4bd22942 to keyring 
D: added key gpg-pubkey-863a853d-4f55f54d to keyring 
D: Using legacy gpg-pubkey(s) from rpmdb 
D: opening db index /var/lib/rpm/Providename rdonly mode=0x0 
D: closed db index /var/lib/rpm/Providename 
D: closed db index /var/lib/rpm/Name 
D: closed db index /var/lib/rpm/Packages 
D: closed db environment /var/lib/rpm 
Loaded plugins: etckeeper, fastestmirror, rhnplugin 
Config time: 0.073 
D: login(forceUpdate=False) invoked 
D: readCachedLogin invoked 
D: Checking pickled loginInfo, currentTime=1422360920.07, createTime=1422360336.74, expire-offset=3600.0 
D: readCachedLogin(): using pickled loginInfo set to expire at 1422363936.74 
D: rpcServer: Calling XMLRPC up2date.listChannels 
This system is receiving updates from RHN Classic or Red Hat Satellite. 
rpmdb time: 0.000 
Loading mirror speeds from cached hostfile 
repo time: 0.002 
Setting up Package Sacks 
pkgsack time: 0.253 
D: local action status: (0, 'rpm database not modified since last update (or package list recently updated)', {}) 
D: rpcServer: Calling XMLRPC registration.welcome_message 
D: closed db index /var/lib/rpm/Providename 
D: closed db index /var/lib/rpm/Name 
D: closed db index /var/lib/rpm/Packages 
D: closed db environment /var/lib/rpm 


2015-01-27 10:53 GMT+00:00 Alexander Innes < senni at necurity.co.uk > : 

<blockquote>

Schedual the action then on the server do rhn-chck -vvv. It will tell you what the client's doing, my gut feeling is either SSL errors OR permsions (rhn-actions-control is it?) 

On 26 January 2015 at 23:16, Chris Swingler < chris at chrisswingler.com > wrote: 

<blockquote>

Hey Spacewalk-list! 

I have quite a few systems (I haven't dug into seeing if there is any consistency in CentOS versions or the RHN tools) that just plain do not seem to get tasks back from the Spacewalk server. We're running Spacewalk 2.2 on CentOS release 6.5 (Final). 

I can create a simple task in Spacewalk, like just print the output of "date", and an rhn_check never seems to execute it. It shows up under Events > Pending, the SystemIDs match, but it never moves. 

Running a packet capture to watch the transaction, it looks like Spacewalk itself isn't even sending the task to the remote system. Spacewalk happily replies with an HTTP 200 and an empty payload. 

HTTP/1.1 200 OK 
Date: Mon, 26 Jan 2015 22:47:55 GMT 
Server: Apache 
Content-Length: 126 
X-RHN-Server-Capability: registration.finish_message(1)=1 
X-RHN-Server-Capability: registration.remaining_subscriptions(1)=1 
X-RHN-Server-Capability: abrt(1)=1 
X-RHN-Server-Capability: registration.update_contact_info(1)=1 
X-RHN-Server-Capability: staging_content(1)=1 
X-RHN-Server-Capability: applet.has_base_channel(1)=1 
X-RHN-Server-Capability: registration.smbios(1)=1 
X-RHN-Server-Capability: registration.extended_update_support(1)=1 
X-RHN-Server-Capability: rhncfg.filetype.directory(1)=1 
X-RHN-Server-Capability: registration.update_systemid(1)=1 
X-RHN-Server-Capability: registration.register_osad(1)=1 
X-RHN-Server-Capability: registration.delta_packages(1)=1 
X-RHN-Server-Capability: cpu_sockets(1)=1 
X-RHN-Server-Capability: ipv6(1)=1 
X-RHN-Server-Capability: rhncfg.content.base64_decode(1)=1 
X-RHN-Server-Capability: xmlrpc.packages.extended_profile(1-2)=1 
X-RHN-Server-Capability: xmlrpc.errata.patch_names(1)=1 
X-RHN-Server-Capability: xmlrpc.packages.checksums(1)=1 
X-RHN-Server-Capability: xmlrpc.login.extra_data(1)=1 
X-RHN-Proxy-Version: 0 
X-Transport-Info: Extended Capabilities Transport (C) Red Hat, Inc (version 2.5.72-1.el6) 
X-RHN-Client-Version: 1 
Connection: close 
Content-Type: text/xml 
<?xml version='1.0'?> 
<methodResponse> 
<params> 
<param> 
<value><string></string></value> 
</param> 
</params> 
</methodResponse> 


Full transaction at 

https://gist.github.com/cswingler/f718abcb9ce290adec29 

rhn_server_xmlrpc.log simply shows: 

2015/01/26 16:52:07 -05:00 30997 172.29.7.14 : xmlrpc/queue.get(1000015056, 2, 'checkins enabled') 
2015/01/26 16:52:07 -05:00 30998 172.29.7.14 : xmlrpc/up2date.listChannels(1000015056,) 
2015/01/26 16:52:08 -05:00 30995 172.29.7.14 : xmlrpc/registration.welcome_message('lang: None',) 

Any basic things I should be checking? I've tried updating the spacewalk/rhn tools on the agent, restarting Spacewalk itself, and nothing seems to make a change. This issue, as far as I can recall, did not seem to appear until after we upgraded to 2.2. 

Some systems seem just fine, and I can't seem to find a consistent reason why some succeed and others fail. 

Thanks! 

_______________________________________________ 
Spacewalk-list mailing list 
Spacewalk-list at redhat.com 
https://www.redhat.com/mailman/listinfo/spacewalk-list 





_______________________________________________ 
Spacewalk-list mailing list 
Spacewalk-list at redhat.com 
https://www.redhat.com/mailman/listinfo/spacewalk-list 

</blockquote>




-- 
Paulo Silva < paulojjs at gmail.com > 

_______________________________________________ 
Spacewalk-list mailing list 
Spacewalk-list at redhat.com 
https://www.redhat.com/mailman/listinfo/spacewalk-list 

</blockquote>



_______________________________________________ 
Spacewalk-list mailing list 
Spacewalk-list at redhat.com 
https://www.redhat.com/mailman/listinfo/spacewalk-list 

</blockquote>




-- 
Paulo Silva < paulojjs at gmail.com > 

_______________________________________________ 
Spacewalk-list mailing list 
Spacewalk-list at redhat.com 
https://www.redhat.com/mailman/listinfo/spacewalk-list 

</blockquote>



_______________________________________________ 
Spacewalk-list mailing list 
Spacewalk-list at redhat.com 
https://www.redhat.com/mailman/listinfo/spacewalk-list 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20150127/4ff5674f/attachment.htm>


More information about the Spacewalk-list mailing list