[Spacewalk-list] Clients not picking up requested changes, period
Chris Swingler
chris at chrisswingler.com
Tue Jan 27 17:36:51 UTC 2015
Ah here we go, just barfed out a traceback and emailed it to me:
Exception reported from spwk-app-chi-01.company.com
Time: Tue Jan 27 11:31:48 2015
Exception type <class 'spacewalk.common.rhnException.rhnException'>
Exception while handling function registration.new_system
Request object information:
URI: /XMLRPC
Remote Host: zbx-prx-elk-04.company.com
Server Name: spacewalk.company.com:80
Headers passed in:
Accept-Encoding: identity
CONTENT_LENGTH: 1577
CONTENT_TYPE: text/xml
DOCUMENT_ROOT: /var/www/html
GATEWAY_INTERFACE: CGI/1.1
HTTP_ACCEPT_ENCODING: identity
HTTP_HOST: spacewalk.company.com
HTTP_USER_AGENT: rhn.rpclib.py/2.5.72-1.el6
HTTP_X_CLIENT_VERSION: 1
HTTP_X_INFO: RPC Processor (C) Red Hat, Inc (version 2.5.72-1.el6)
HTTP_X_RHN_CLIENT_CAPABILITY:
osad.ping(1)=1,packages.verify(1)=1,packages.extended_profile(2)=1,osad.rhn_check(1)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1
HTTP_X_RHN_TRANSPORT_CAPABILITY: follow-redirects=3
HTTP_X_TRANSPORT_INFO: Extended Capabilities Transport (C) Red Hat, Inc
(version 2.5.72-1.el6)
HTTP_X_UP2DATE_VERSION: 2.2.7-1.el6
Host: spacewalk.company.com
PATH_INFO:
QUERY_STRING:
REMOTE_ADDR: 172.29.7.14
REMOTE_PORT: 36965
REQUEST_METHOD: POST
REQUEST_URI: /XMLRPC
SCRIPT_FILENAME: /usr/share/rhn/wsgi/xmlrpc.py
SCRIPT_NAME: /XMLRPC
SCRIPT_URI: http://spacewalk.company.com/XMLRPC
SCRIPT_URL: /XMLRPC
SERVER_ADDR: 172.31.5.190
SERVER_ADMIN: root at localhost
SERVER_NAME: spacewalk.company.com
SERVER_PORT: 80
SERVER_PROTOCOL: HTTP/1.1
SERVER_SIGNATURE: <address>Apache Server at spacewalk.company.com Port
80</address>
SERVER_SOFTWARE: Apache
User-Agent: rhn.rpclib.py/2.5.72-1.el6
X-Client-Version: 1
X-Info: RPC Processor (C) Red Hat, Inc (version 2.5.72-1.el6)
X-RHN-Client-Capability:
osad.ping(1)=1,packages.verify(1)=1,packages.extended_profile(2)=1,osad.rhn_check(1)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1
X-RHN-Transport-Capability: follow-redirects=3
X-Transport-Info: Extended Capabilities Transport (C) Red Hat, Inc (version
2.5.72-1.el6)
X-Up2Date-Version: 2.2.7-1.el6
mod_ssl.is_https: <built-in method ssl_is_https of mod_wsgi.Adapter object
at 0x7f535b73d030>
mod_ssl.var_lookup: <built-in method ssl_var_lookup of mod_wsgi.Adapter
object at 0x7f535b73d030>
mod_wsgi.application_group: spwk-app-chi-01.company.com|/xmlrpc
mod_wsgi.callable_object: application
mod_wsgi.handler_script:
mod_wsgi.input_chunked: 0
mod_wsgi.listener_host:
mod_wsgi.listener_port: 80
mod_wsgi.process_group:
mod_wsgi.request_handler: wsgi-script
mod_wsgi.script_reloading: 1
mod_wsgi.version: (3, 2)
wsgi.errors: <mod_wsgi.Log object at 0x7f535b6c3f70>
wsgi.file_wrapper: <built-in method file_wrapper of mod_wsgi.Adapter object
at 0x7f535b73d030>
wsgi.input: <mod_wsgi.Input object at 0x7f535b6c3f30>
wsgi.multiprocess: True
wsgi.multithread: False
wsgi.run_once: False
wsgi.url_scheme: http
wsgi.version: (1, 1)
Exception Handler Information
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/spacewalk/server/apacheRequest.py",
line 122, in call_function
response = func(*params)
File "/usr/share/rhn/server/handlers/xmlrpc/registration.py", line 511, in
new_system
architecture, data)
File "/usr/share/rhn/server/handlers/xmlrpc/registration.py", line 300, in
create_system
newserv.user.reload(newserv.server['creator_id'])
File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnUser.py", line
246, in reload
raise rhnException("Could not find contact record for id", user_id)
rhnException: ('Could not find contact record for id', None)
Local variables by frame
Frame reload in
/usr/lib/python2.6/site-packages/spacewalk/server/rhnUser.py at line 246
self = <type 'instance'> <spacewalk.server.rhnUser.User
instance at 0x7f535ecc35a8>
user_id = <type 'NoneType'> None
Frame create_system in
/usr/share/rhn/server/handlers/xmlrpc/registration.py at line 300
release_version = <type 'str'> 6
log_user_id = <type 'int'> 286
user = <type 'NoneType'> None
log_item = <type 'str'> token =
'9b601306ac1616daf1510d93a5084bf6'
self = <type 'instance'>
<server.handlers.xmlrpc.registration.Registration instance at
0x7f535e378d88>
applet_uuid = <type 'NoneType'> None
profile_name = <type 'str'> zbx-prx-elk-04.company.com
channel = <type 'NoneType'> None
release = <type 'str'> 6
token_string = <type 'str'> 9b601306ac1616daf1510d93a5084bf6
up2date_uuid = <type 'NoneType'> None
data = <type 'dict'> {'token':
'9b601306ac1616daf1510d93a5084bf6', 'profile_name': '
zbx-prx-elk-04.company.com', 'architecture': 'x86_64', 'os_release': '6',
'smbios': {'smbios.system.serial': 'VMware-42 2f 69 71 4c 96 98 f5-53 f3 3b
d4 cf 45 de c2', 'smbios.system.product': 'VMware Virtual Platform',
'smbios.system.family': 'Not Specified', 'smbios.bios.vendor': 'Phoenix
Technologies LTD', 'smbios.system.manufacturer': 'VMware, Inc.',
'smbios.system.skunumber': 'Not Specified', 'smbios.system.version':
'None', 'smbios.system.uuid': '422f6971-4c96-98f5-53f3-3bd4cf45dec2'},
'release_name': 'centos-release'}
newserv = <class
'spacewalk.server.rhnServer.server_class.Server'> <Server Class at
139996049716624: {'self.cert': None, 'self.server': {'info': (None, 0),
'creator_id': (None, 0), 'provision_state_id': (None, 0), 'name': ('
zbx-prx-elk-04.company.com', 0), 'created': (datetime.datetime(2013, 10,
17, 12, 46, 48, 613614, tzinfo=<psycopg2.tz.FixedOffsetTimezone object at
0x7f535ed03f90>), 0), 'auto_update': ('N', 0), 'running_kernel':
('2.6.32-358.2.1.el6.x86_64', 0), 'description': ('Initial Registration
Parameters:\nOS: centos-release\nRelease: 6\nCPU Arch: x86_64', 0),
'org_id': (1, 0), 'modified': (datetime.datetime(2015, 1, 1, 1, 4, 12,
661434, tzinfo=<psycopg2.tz.FixedOffsetTimezone object at 0x7f535ed03b10>),
0), 'server_arch_id': (1017, 0), 'secret':
('34da7b20215c6c8d68bd68bbfafdcb54', 0), 'last_boot': (1389283343.6636419,
0), 'digital_server_id': ('ID-1000015056', 0), 'release': ('6', 0),
'channels_changed': (datetime.datetime(2014, 11, 17, 11, 37, 28,
508193, tzinfo=<psycopg2.tz.FixedOffsetTimezone object at 0x7f535ed03fd0>),
0), 'os': ('centos-release', 0), 'id': (1000015056, 0), 'cobbler_id':
(None, 0)}}>
tokens_obj = <type 'instance'>
<spacewalk.server.rhnServer.server_token.ReRegistrationToken instance at
0x7f535ecbdb00>
architecture = <type 'str'> x86_64
Frame new_system in /usr/share/rhn/server/handlers/xmlrpc/registration.py
at line 511
self = <type 'instance'>
<server.handlers.xmlrpc.registration.Registration instance at
0x7f535e378d88>
release_version = <type 'str'> 6
profile_name = <type 'str'> zbx-prx-elk-04.company.com
architecture = <type 'str'> x86_64
data = <type 'dict'> {'token':
'9b601306ac1616daf1510d93a5084bf6', 'profile_name': '
zbx-prx-elk-04.company.com', 'architecture': 'x86_64', 'os_release': '6',
'smbios': {'smbios.system.serial': 'VMware-42 2f 69 71 4c 96 98 f5-53 f3 3b
d4 cf 45 de c2', 'smbios.system.product': 'VMware Virtual Platform',
'smbios.system.family': 'Not Specified', 'smbios.bios.vendor': 'Phoenix
Technologies LTD', 'smbios.system.manufacturer': 'VMware, Inc.',
'smbios.system.skunumber': 'Not Specified', 'smbios.system.version':
'None', 'smbios.system.uuid': '422f6971-4c96-98f5-53f3-3bd4cf45dec2'},
'release_name': 'centos-release'}
user = <type 'NoneType'> None
Frame call_function in
/usr/lib/python2.6/site-packages/spacewalk/server/apacheRequest.py at line
173
e = <class 'spacewalk.common.rhnException.rhnException'>
('Could not find contact record for id', None)
self = <type 'instance'>
<spacewalk.server.apacheRequest.apachePOST instance at 0x7f535ebdcf80>
force_rollback = <type 'int'> 1
params = <type 'tuple'> ({'token':
'9b601306ac1616daf1510d93a5084bf6', 'profile_name': '
zbx-prx-elk-04.company.com', 'architecture': 'x86_64', 'os_release': '6',
'smbios': {'smbios.system.serial': 'VMware-42 2f 69 71 4c 96 98 f5-53 f3 3b
d4 cf 45 de c2', 'smbios.system.product': 'VMware Virtual Platform',
'smbios.system.family': 'Not Specified', 'smbios.bios.vendor': 'Phoenix
Technologies LTD', 'smbios.system.manufacturer': 'VMware, Inc.',
'smbios.system.skunumber': 'Not Specified', 'smbios.system.version':
'None', 'smbios.system.uuid': '422f6971-4c96-98f5-53f3-3bd4cf45dec2'},
'release_name': 'centos-release'},)
func = <type 'instancemethod'> <bound method
Registration.new_system of
<server.handlers.xmlrpc.registration.Registration instance at
0x7f535e378d88>>
method = <type 'unicode'> registration.new_system
Frame process in
/usr/lib/python2.6/site-packages/spacewalk/server/apacheRequest.py at line
484
fd = <type 'instance'> <cStringIO.StringO object at
0x7f535ecacf10>
_body = <type 'str'> <?xml version='1.0'?>
<methodCall>
<methodName>registration.new_system</methodName>
<params>
<param>
<value><struct>
<member>
<name>token</name>
<value><string>9b601306ac1616daf1510d93a5084bf6</string></value>
</member>
<member>
<name>profile_name</name>
<value><string>zbx-prx-elk-04.company.com</string></value>
</member>
<member>
<name>architecture</name>
<value><string>x86_64</string></value>
</member>
<member>
<name>os_release</name>
<value><string>6</string></value>
</member>
<member>
<name>smbios</name>
<value><struct>
<member>
<name>smbios.system.serial</name>
<value><string>VMware-42 2f 69 71 4c 96 98 f5-53 f3 3b d4 cf 45 de
c2</string></value>
</member>
<member>
<name>smbios.system.product</name>
<value><string>VMware Virtual Platform</string></value>
</member>
<member>
<name>smbios.system.family</name>
<value><string>Not Specified</string></value>
</member>
<member>
<name>smbios.bios.vendor</name>
<value><string>Phoenix Technologies LTD</string></value>
</member>
<member>
<name>smbios.system.manufacturer</name>
<value><string>VMware, Inc.</string></value>
</member>
<member>
<name>smbios.system.skunumber</name>
<value><string>Not Specified</string></value>
</member>
<member>
<name>smbios.system.version</name>
<value><string>None</string></value>
</member>
<member>
<name>smbios.system.uuid</name>
<value><string>422f6971-4c96-98f5-53f3-3bd4cf45dec2</string></value>
</member>
</struct></value>
</member>
<member>
<name>release_name</name>
<value><string>centos-release</string></value>
</member>
</struct></value>
</param>
</params>
</methodCall>
params = <type 'tuple'> ({'token':
'9b601306ac1616daf1510d93a5084bf6', 'profile_name': '
zbx-prx-elk-04.company.com', 'architecture': 'x86_64', 'os_release': '6',
'smbios': {'smbios.system.serial': 'VMware-42 2f 69 71 4c 96 98 f5-53 f3 3b
d4 cf 45 de c2', 'smbios.system.product': 'VMware Virtual Platform',
'smbios.system.family': 'Not Specified', 'smbios.bios.vendor': 'Phoenix
Technologies LTD', 'smbios.system.manufacturer': 'VMware, Inc.',
'smbios.system.skunumber': 'Not Specified', 'smbios.system.version':
'None', 'smbios.system.uuid': '422f6971-4c96-98f5-53f3-3bd4cf45dec2'},
'release_name': 'centos-release'},)
method = <type 'unicode'> registration.new_system
self = <type 'instance'>
<spacewalk.server.apacheRequest.apachePOST instance at 0x7f535ebdcf80>
Frame handler in
/usr/lib/python2.6/site-packages/spacewalk/server/apacheHandler.py at line
196
h = <type 'instance'>
<spacewalk.server.rhnSQL.driver_postgresql.Cursor instance at
0x7f535e3729e0>
self = <type 'instance'>
<spacewalk.server.apacheHandler.apacheHandler instance at 0x7f535e6e24d0>
req = <type 'instance'> <wsgi.wsgiRequest.WsgiRequest
instance at 0x7f535e36bdd0>
ret = <type 'int'> 0
templateStrings = <type 'dict'> {'email_account_info': '\nAccount
Information:\n Your Spacewalk login: <login />\n Your Spacewalk
email address: <email-address />', 'email_footer': '--the Spacewalk Team',
'hostname': 'crp-spwk-chi-02.company.com'}
row = <type 'NoneType'> None
Frame handle in /usr/share/rhn/wsgi/wsgiHandler.py at line 45
replacements = <type 'dict'> {'Xml': 'XML', 'Actualuri':
'ActualURI', 'Md5Sum': 'MD5sum', '_': '-', 'Rhn': 'RHN'}
component_type = <type 'str'> server.xmlrpc
start_response = <type 'builtin_function_or_method'> <built-in method
start_response of mod_wsgi.Adapter object at 0x7f535b73d030>
k = <type 'str'> Rhn
req = <type 'instance'> <wsgi.wsgiRequest.WsgiRequest
instance at 0x7f535e36bdd0>
parseServ = <type 'instancemethod'> <bound method
apacheHandler.headerParserHandler of
<spacewalk.server.apacheHandler.apacheHandler instance at 0x7f535e6e24d0>>
ret = <type 'int'> 0
server = <type 'str'> xmlrpc
environ = <type 'dict'> {'mod_wsgi.listener_port': '80',
'mod_ssl.var_lookup': <built-in method ssl_var_lookup of mod_wsgi.Adapter
object at 0x7f535b73d030>, 'X-RHN-Client-Capability':
'osad.ping(1)=1,packages.verify(1)=1,packages.extended_profile(2)=1,osad.rhn_check(1)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1',
'SERVER_SOFTWARE': 'Apache', 'SCRIPT_NAME': '/XMLRPC',
'HTTP_X_TRANSPORT_INFO': 'Extended Capabilities Transport (C) Red Hat, Inc
(version 2.5.72-1.el6)', 'mod_wsgi.handler_script': '', 'SERVER_SIGNATURE':
'<address>Apache Server at spacewalk.company.com Port 80</address>\n',
'REQUEST_METHOD': 'POST', 'PATH_INFO': '', 'SERVER_PROTOCOL': 'HTTP/1.1',
'QUERY_STRING': '', 'X-Info': 'RPC Processor (C) Red Hat, Inc (version
2.5.72-1.el6)', 'CONTEN
T_LENGTH': '1577', 'HTTP_USER_AGENT': 'rhn.rpclib.py/2.5.72-1.el6',
'HTTP_X_UP2DATE_VERSION': '2.2.7-1.el6', 'SERVER_NAME': '
spacewalk.company.com', 'REMOTE_ADDR': '172.29.7.14',
'mod_wsgi.request_handler': 'wsgi-script', 'User-Agent': '
rhn.rpclib.py/2.5.72-1.el6', 'wsgi.url_scheme': 'http', 'Host': '
spacewalk.company.com', 'X-RHN-Transport-Capability': 'follow-redirects=3',
'mod_wsgi.callable_object': 'application', 'SERVER_PORT': '80',
'wsgi.multiprocess': True, 'mod_wsgi.input_chunked': '0', 'SERVER_ADDR':
'172.31.5.190', 'DOCUMENT_ROOT': '/var/www/html', 'mod_wsgi.process_group':
'', 'Accept-Encoding': 'identity', 'HTTP_X_RHN_CLIENT_CAPABILITY':
'osad.ping(1)=1,packages.verify(1)=1,packages.extended_profile(2)=1,osad.rhn_check(1)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTr
ansaction(1)=1', 'SCRIPT_FILENAME': '/usr/share/rhn/wsgi/xmlrpc.py',
'SERVER_ADMIN': 'root at localhost', 'X-Up2Date-Version': '2.2.7-1.el6',
'X-Transport-Info': 'Extended Capabilities Transport (C) Red Hat, Inc
(version 2.5.72-1.el6)', 'SCRIPT_URI': 'http://spacewalk.company.com/XMLRPC',
'HTTP_X_CLIENT_VERSION': '1', 'wsgi.input': <mod_wsgi.Input object at
0x7f535b6c3f30>, 'HTTP_HOST': 'spacewalk.company.com', 'SCRIPT_URL':
'/XMLRPC', 'wsgi.multithread': False, 'REQUEST_URI': '/XMLRPC',
'X-Client-Version': '1', 'wsgi.version': (1, 1), 'GATEWAY_INTERFACE':
'CGI/1.1', 'wsgi.run_once': False, 'wsgi.errors': <mod_wsgi.Log object at
0x7f535b6c3f70>, 'REMOTE_PORT': '36965', 'mod_wsgi.listener_host': '',
'mod_wsgi.version': (3, 2), 'HTTP_X_RHN_TRANSPORT_CAPABILITY':
'follow-redirects=3', 'mod_ssl.is_https': <built-in method ssl_is_https of
mod_wsgi.Adapter object at 0x7f535b73d030>, 'CONTENT_TYPE': 'text/xml',
'mod_wsgi.application_group': 'spwk-app-chi-01.company.com|/xmlrpc',
'mod_wsgi.script_reloading': '1', 'wsgi.file_wrapper': <built-in method
file_wrapper of mod_wsgi.Adapter object at 0x7f535b73d030>,
'HTTP_ACCEPT_ENCODING': 'identity', 'HTTP_X_INFO': 'RPC Processor (C) Red
Hat, Inc (version 2.5.72-1.el6)'}
key = <type 'str'> HTTP_X_INFO
v = <type 'str'> RHN
servertype = <type 'str'> spacewalk.server.apacheServer
new_key = <type 'str'> X-Info
appServ = <type 'instancemethod'> <bound method
apacheHandler.handler of <spacewalk.server.apacheHandler.apacheHandler
instance at 0x7f535e6e24d0>>
Frame application in /usr/share/rhn/wsgi/xmlrpc.py at line 22
start_response = <type 'builtin_function_or_method'> <built-in method
start_response of mod_wsgi.Adapter object at 0x7f535b73d030>
environ = <type 'dict'> {'mod_wsgi.listener_port': '80',
'mod_ssl.var_lookup': <built-in method ssl_var_lookup of mod_wsgi.Adapter
object at 0x7f535b73d030>, 'X-RHN-Client-Capability':
'osad.ping(1)=1,packages.verify(1)=1,packages.extended_profile(2)=1,osad.rhn_check(1)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTransaction(1)=1',
'SERVER_SOFTWARE': 'Apache', 'SCRIPT_NAME': '/XMLRPC',
'HTTP_X_TRANSPORT_INFO': 'Extended Capabilities Transport (C) Red Hat, Inc
(version 2.5.72-1.el6)', 'mod_wsgi.handler_script': '', 'SERVER_SIGNATURE':
'<address>Apache Server at spacewalk.company.com Port 80</address>\n',
'REQUEST_METHOD': 'POST', 'PATH_INFO': '', 'SERVER_PROTOCOL': 'HTTP/1.1',
'QUERY_STRING': '', 'X-Info': 'RPC Processor (C) Red Hat, Inc (version
2.5.72-1.el6)', 'CONTEN
T_LENGTH': '1577', 'HTTP_USER_AGENT': 'rhn.rpclib.py/2.5.72-1.el6',
'HTTP_X_UP2DATE_VERSION': '2.2.7-1.el6', 'SERVER_NAME': '
spacewalk.company.com', 'REMOTE_ADDR': '172.29.7.14',
'mod_wsgi.request_handler': 'wsgi-script', 'User-Agent': '
rhn.rpclib.py/2.5.72-1.el6', 'wsgi.url_scheme': 'http', 'Host': '
spacewalk.company.com', 'X-RHN-Transport-Capability': 'follow-redirects=3',
'mod_wsgi.callable_object': 'application', 'SERVER_PORT': '80',
'wsgi.multiprocess': True, 'mod_wsgi.input_chunked': '0', 'SERVER_ADDR':
'172.31.5.190', 'DOCUMENT_ROOT': '/var/www/html', 'mod_wsgi.process_group':
'', 'Accept-Encoding': 'identity', 'HTTP_X_RHN_CLIENT_CAPABILITY':
'osad.ping(1)=1,packages.verify(1)=1,packages.extended_profile(2)=1,osad.rhn_check(1)=1,configfiles.base64_enc(1)=1,reboot.reboot(1)=1,configfiles.deploy(1)=1,caneatCheese(1)=1,packages.rollBack(1)=1,configfiles.mtime_upload(1)=1,configfiles.diff(1)=1,packages.update(2)=2,script.run(1)=1,configfiles.upload(1)=1,packages.runTr
ansaction(1)=1', 'SCRIPT_FILENAME': '/usr/share/rhn/wsgi/xmlrpc.py',
'SERVER_ADMIN': 'root at localhost', 'X-Up2Date-Version': '2.2.7-1.el6',
'X-Transport-Info': 'Extended Capabilities Transport (C) Red Hat, Inc
(version 2.5.72-1.el6)', 'SCRIPT_URI': 'http://spacewalk.company.com/XMLRPC',
'HTTP_X_CLIENT_VERSION': '1', 'wsgi.input': <mod_wsgi.Input object at
0x7f535b6c3f30>, 'HTTP_HOST': 'spacewalk.company.com', 'SCRIPT_URL':
'/XMLRPC', 'wsgi.multithread': False, 'REQUEST_URI': '/XMLRPC',
'X-Client-Version': '1', 'wsgi.version': (1, 1), 'GATEWAY_INTERFACE':
'CGI/1.1', 'wsgi.run_once': False, 'wsgi.errors': <mod_wsgi.Log object at
0x7f535b6c3f70>, 'REMOTE_PORT': '36965', 'mod_wsgi.listener_host': '',
'mod_wsgi.version': (3, 2), 'HTTP_X_RHN_TRANSPORT_CAPABILITY':
'follow-redirects=3', 'mod_ssl.is_https': <built-in method ssl_is_https of
mod_wsgi.Adapter object at 0x7f535b73d030>, 'CONTENT_TYPE': 'text/xml',
'mod_wsgi.application_group': 'spwk-app-chi-01.company.com|/xmlrpc',
'mod_wsgi.script_reloading': '1', 'wsgi.file_wrapper': <built-in method
file_wrapper of mod_wsgi.Adapter object at 0x7f535b73d030>,
'HTTP_ACCEPT_ENCODING': 'identity', 'HTTP_X_INFO': 'RPC Processor (C) Red
Hat, Inc (version 2.5.72-1.el6)'}
Environment for PID=8635 on exception:
CONSOLETYPE = vt
LANG = C
LANGSH_SOURCED = 1
PATH = /sbin:/usr/sbin:/bin:/usr/bin
PREVLEVEL = N
PWD = /
RUNLEVEL = 3
SHLVL = 3
TERM = linux
UPSTART_EVENTS = runlevel
UPSTART_INSTANCE =
UPSTART_JOB = rc
_ = /usr/sbin/httpd
previous = N
runlevel = 3
So I'm thinking there's something funky with data consistency in the
database for these hosts. Not a clue what it is, or what caused it, though.
On Tue, Jan 27, 2015 at 10:48 AM, Chris Swingler <chris at chrisswingler.com>
wrote:
> More interesting things:
>
> Using the reactivation feature doesn't work.
>
> [root at zbx-prx-elk-04 log]# /usr/sbin/rhnreg_ks --force --serverUrl=
> http://spacewalk.companyname.com/XMLRPC
> --activationkey=9b601306ac1616daf1510d93a5084bf6
> Error communicating with server. The message was:
> Internal Server Error
>
> Give it the wrong token, and you get an actual error:
>
> [root at zbx-prx-elk-04 log]# /usr/sbin/rhnreg_ks --force --serverUrl=
> http://spacewalk.companyname.com/XMLRPC
> --activationkey=1-9b601306ac1616daf1510d93a5084bf6
>
> Error Message:
> Could not find token '1-9b601306ac1616daf1510d93a5084bf6'
> Error Class Code: 60
> Error Class Info:
> The activation token specified could not be found on the server.
> Please retry with a valid key.
>
>
>
> On Tue, Jan 27, 2015 at 10:36 AM, Chris Swingler <chris at chrisswingler.com>
> wrote:
>
>> Yeah, osad/rhnsd don't seem to have any bearing on this problem, turning
>> them on/off and trying again was one of the first things I tried. No
>> change; so it's not like osad is somehow half-picking up the task and then
>> not finishing it.
>>
>> Worth mentioning, if it's not clear: Spacewalk never sees the task even
>> get acknowledged by the remote system. It just stays at:
>>
>> This action will be executed after 1/27/15 10:28:00 AM CST
>> This action's status is: Queued.
>> This action has not yet been picked up.
>>
>> Of course, brand new VMs I set up to try to replicate this issue work
>> fine.
>>
>> Paulo: That's an idea, but not necessarily a *good* one... :)
>>
>> On Tue, Jan 27, 2015 at 10:29 AM, Paulo Silva <paulojjs at gmail.com> wrote:
>>
>>> I've found an ugly fix for this, all I had to do was to use rhnreg_ks to
>>> re-register the system and now events on the new profile are being picked
>>> up by rhn_check.
>>>
>>> 2015-01-27 16:07 GMT+00:00 Paulo Silva <paulojjs at gmail.com>:
>>>
>>>> Hi Glen,
>>>>
>>>> Can't try that since I'm using rhnsd and not osad.
>>>>
>>>> Thanks
>>>>
>>>> 2015-01-27 15:28 GMT+00:00 Glen Collins <glenc2004 at comcast.net>:
>>>>
>>>>> Hi Paulo,
>>>>>
>>>>> Why don't you try this on the client....
>>>>>
>>>>> service osad stop
>>>>> rm -rf /etc/sysconfig/rhn/osad-auth.conf
>>>>> service osad start
>>>>> (Wait for 2 minutes) and run (On the client)
>>>>> /usr/sbin/rhn_check
>>>>>
>>>>> I've had the same issues and just restarting OSAD didn't do it for me.
>>>>> But when I removed the auth file, that seemed to fix things. Not elegant in
>>>>> any sort of means but it seems to work. Have not tried it long term since
>>>>> I'm putting this "fix" into place right now. I guess this is what happens
>>>>> when you cobble a product together and "make it work". I have found jabber
>>>>> to be a PITA!
>>>>>
>>>>> Anyways, hope this helps!
>>>>>
>>>>> Glen Collins
>>>>>
>>>>> ------------------------------
>>>>> It's not permissions:
>>>>>
>>>>> [root at zbx-prx-elk-04 ~]# rhn-actions-control --report
>>>>> deploy is enabled
>>>>> diff is enabled
>>>>> upload is enabled
>>>>> mtime_upload is enabled
>>>>> run is enabled
>>>>>
>>>>> It's not SSL either, we have our systems configured to speak in the
>>>>> clear while troubleshooting this issue. Flipping over to HTTPS doesn't
>>>>> change anything.
>>>>>
>>>>> This particular server has three tasks pending now: two config file
>>>>> pushes and one remote command (just to run "date"). Here's the output of
>>>>> rhn_check -vvvv:
>>>>>
>>>>> [root at zbx-prx-elk-04 ~]# rhn_check -vvvv
>>>>> D: opening db environment /var/lib/rpm cdb:mpool:joinenv
>>>>> D: opening db index /var/lib/rpm/Packages rdonly mode=0x0
>>>>> D: locked db index /var/lib/rpm/Packages
>>>>> D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key
>>>>> D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key
>>>>> D: loading keyring from rpmdb
>>>>> D: opening db index /var/lib/rpm/Name rdonly mode=0x0
>>>>> D: added key gpg-pubkey-b6f8c6bf-48853124 to keyring
>>>>> D: added key gpg-pubkey-8db2536f-4ade0b9d to keyring
>>>>> D: added key gpg-pubkey-66fd4949-4803fe57 to keyring
>>>>> D: added key gpg-pubkey-c105b9de-4e0fd3a3 to keyring
>>>>> D: added key gpg-pubkey-217521f6-45e8a532 to keyring
>>>>> D: added key gpg-pubkey-0608b895-4bd22942 to keyring
>>>>> D: added key gpg-pubkey-863a853d-4f55f54d to keyring
>>>>> D: added key gpg-pubkey-79ea5ed4-508d25a6 to keyring
>>>>> D: Using legacy gpg-pubkey(s) from rpmdb
>>>>> D: opening db index /var/lib/rpm/Providename rdonly mode=0x0
>>>>> D: do_call packages.checkNeedUpdate('rhnsd=1',){}
>>>>> D: opening db environment /var/lib/rpm cdb:mpool:joinenv
>>>>> D: opening db index /var/lib/rpm/Packages rdonly mode=0x0
>>>>> D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key
>>>>> D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key
>>>>> D: loading keyring from rpmdb
>>>>> D: opening db index /var/lib/rpm/Name rdonly mode=0x0
>>>>> D: added key gpg-pubkey-b6f8c6bf-48853124 to keyring
>>>>> D: added key gpg-pubkey-8db2536f-4ade0b9d to keyring
>>>>> D: added key gpg-pubkey-66fd4949-4803fe57 to keyring
>>>>> D: added key gpg-pubkey-c105b9de-4e0fd3a3 to keyring
>>>>> D: added key gpg-pubkey-217521f6-45e8a532 to keyring
>>>>> D: added key gpg-pubkey-0608b895-4bd22942 to keyring
>>>>> D: added key gpg-pubkey-863a853d-4f55f54d to keyring
>>>>> D: added key gpg-pubkey-79ea5ed4-508d25a6 to keyring
>>>>> D: Using legacy gpg-pubkey(s) from rpmdb
>>>>> D: opening db index /var/lib/rpm/Providename rdonly mode=0x0
>>>>> D: closed db index /var/lib/rpm/Providename
>>>>> D: closed db index /var/lib/rpm/Name
>>>>> D: closed db index /var/lib/rpm/Packages
>>>>> D: closed db environment /var/lib/rpm
>>>>> Loaded plugins: fastestmirror, rhnplugin
>>>>> Config time: 0.040
>>>>> D: login(forceUpdate=False) invoked
>>>>> D: readCachedLogin invoked
>>>>> D: Checking pickled loginInfo, currentTime=1422370609.99,
>>>>> createTime=1422370587.99, expire-offset=3600.0
>>>>> D: readCachedLogin(): using pickled loginInfo set to expire at
>>>>> 1422374187.99
>>>>> D: rpcServer: Calling XMLRPC up2date.listChannels
>>>>> Setting up Package Sacks
>>>>> Loading mirror speeds from cached hostfile
>>>>> pkgsack time: 0.071
>>>>> rpmdb time: 0.000
>>>>> Loading mirror speeds from cached hostfile
>>>>> repo time: 0.000
>>>>> D: local action status: (0, 'rpm database not modified since last
>>>>> update (or package list recently updated)', {})
>>>>> D: rpcServer: Calling XMLRPC registration.welcome_message
>>>>> D: closed db index /var/lib/rpm/Providename
>>>>> D: closed db index /var/lib/rpm/Name
>>>>> D: closed db index /var/lib/rpm/Packages
>>>>> D: closed db environment /var/lib/rpm
>>>>>
>>>>> Paulo: To answer your question about the API, spacecmd calls it
>>>>> extensively; and you can look up pending tasks by ID.
>>>>>
>>>>> spacecmd {SSM:0}> schedule_details 578593
>>>>> ID: 578593
>>>>> Action: Remote Command on zbx-prx-elk-04.company.com.
>>>>> User: admincswingler
>>>>> Date: 20150126T16:36:00
>>>>>
>>>>> Completed: 0
>>>>> Failed: 0
>>>>> Pending: 1
>>>>>
>>>>> Pending Systems
>>>>> ---------------
>>>>> zbx-prx-elk-04.company.com
>>>>> spacecmd {SSM:0}> schedule_details 578588
>>>>> ID: 578588
>>>>> Action: Deploy config files to system
>>>>> User: admincswingler
>>>>> Date: 20150126T16:22:00
>>>>>
>>>>> Completed: 0
>>>>> Failed: 0
>>>>> Pending: 1
>>>>>
>>>>> Pending Systems
>>>>> ---------------
>>>>> zbx-prx-elk-04.company.com
>>>>> spacecmd {SSM:0}> schedule_details 578586
>>>>> ID: 578586
>>>>> Action: Deploy config files to system
>>>>> User: admincswingler
>>>>> Date: 20150126T16:18:00
>>>>>
>>>>> Completed: 0
>>>>> Failed: 0
>>>>> Pending: 1
>>>>>
>>>>> Pending Systems
>>>>> ---------------
>>>>> zbx-prx-elk-04.company.com
>>>>> spacecmd {SSM:0}>
>>>>>
>>>>> Still scratching my head on this one, and getting rather frustrated :)
>>>>>
>>>>> On Tue, Jan 27, 2015 at 7:02 AM, Paulo Silva <paulojjs at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I'm checking pending events using the web interface (URL
>>>>>> /rhn/systems/details/history/Pending.do?sid=1000010296) and it shows 2
>>>>>> pending events: and Hardware Refresh and an Errata Update.
>>>>>>
>>>>>> The sid seams to be correct with the server I'm running my commands:
>>>>>>
>>>>>> # grep 1000010296 /etc/sysconfig/rhn/systemid
>>>>>> <value><string>ID-1000010296</string></value>
>>>>>>
>>>>>> Is there a way to check for pending events using the API?
>>>>>>
>>>>>> 2015-01-27 12:45 GMT+00:00 Alexander Innes <senni at necurity.co.uk>:
>>>>>>
>>>>>>> Just to make sure im not thinking of the wrong feture are you trying
>>>>>>> to use to remote commands?
>>>>>>> Systems -> System -> somethin -> remote command (or a different one
>>>>>>> so i can test in our environment :) )
>>>>>>>
>>>>>>> from that output its not seeing anything as you said :o
>>>>>>>
>>>>>>> On 27 January 2015 at 12:16, Paulo Silva <paulojjs at gmail.com> wrote:
>>>>>>>
>>>>>>>> I'm having the same issue, running rhn_check does this:
>>>>>>>>
>>>>>>>> # rhn_check -vvv
>>>>>>>> D: opening db environment /var/lib/rpm cdb:mpool:joinenv
>>>>>>>> D: opening db index /var/lib/rpm/Packages rdonly mode=0x0
>>>>>>>> D: locked db index /var/lib/rpm/Packages
>>>>>>>> D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key
>>>>>>>> D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key
>>>>>>>> D: loading keyring from rpmdb
>>>>>>>> D: opening db index /var/lib/rpm/Name rdonly mode=0x0
>>>>>>>> D: added key gpg-pubkey-c105b9de-4e0fd3a3 to keyring
>>>>>>>> D: added key gpg-pubkey-b5c61460-41667588 to keyring
>>>>>>>> D: added key gpg-pubkey-6b8d79e6-3f49313d to keyring
>>>>>>>> D: added key gpg-pubkey-0608b895-4bd22942 to keyring
>>>>>>>> D: added key gpg-pubkey-863a853d-4f55f54d to keyring
>>>>>>>> D: Using legacy gpg-pubkey(s) from rpmdb
>>>>>>>> D: opening db index /var/lib/rpm/Providename rdonly mode=0x0
>>>>>>>> D: do_call packages.checkNeedUpdate('rhnsd=1',){}
>>>>>>>> D: opening db environment /var/lib/rpm cdb:mpool:joinenv
>>>>>>>> D: opening db index /var/lib/rpm/Packages rdonly mode=0x0
>>>>>>>> D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key
>>>>>>>> D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key
>>>>>>>> D: loading keyring from rpmdb
>>>>>>>> D: opening db index /var/lib/rpm/Name rdonly mode=0x0
>>>>>>>> D: added key gpg-pubkey-c105b9de-4e0fd3a3 to keyring
>>>>>>>> D: added key gpg-pubkey-b5c61460-41667588 to keyring
>>>>>>>> D: added key gpg-pubkey-6b8d79e6-3f49313d to keyring
>>>>>>>> D: added key gpg-pubkey-0608b895-4bd22942 to keyring
>>>>>>>> D: added key gpg-pubkey-863a853d-4f55f54d to keyring
>>>>>>>> D: Using legacy gpg-pubkey(s) from rpmdb
>>>>>>>> D: opening db index /var/lib/rpm/Providename rdonly mode=0x0
>>>>>>>> D: closed db index /var/lib/rpm/Providename
>>>>>>>> D: closed db index /var/lib/rpm/Name
>>>>>>>> D: closed db index /var/lib/rpm/Packages
>>>>>>>> D: closed db environment /var/lib/rpm
>>>>>>>> Loaded plugins: etckeeper, fastestmirror, rhnplugin
>>>>>>>> Config time: 0.073
>>>>>>>> D: login(forceUpdate=False) invoked
>>>>>>>> D: readCachedLogin invoked
>>>>>>>> D: Checking pickled loginInfo, currentTime=1422360920.07,
>>>>>>>> createTime=1422360336.74, expire-offset=3600.0
>>>>>>>> D: readCachedLogin(): using pickled loginInfo set to expire at
>>>>>>>> 1422363936.74
>>>>>>>> D: rpcServer: Calling XMLRPC up2date.listChannels
>>>>>>>> This system is receiving updates from RHN Classic or Red Hat
>>>>>>>> Satellite.
>>>>>>>> rpmdb time: 0.000
>>>>>>>> Loading mirror speeds from cached hostfile
>>>>>>>> repo time: 0.002
>>>>>>>> Setting up Package Sacks
>>>>>>>> pkgsack time: 0.253
>>>>>>>> D: local action status: (0, 'rpm database not modified since last
>>>>>>>> update (or package list recently updated)', {})
>>>>>>>> D: rpcServer: Calling XMLRPC registration.welcome_message
>>>>>>>> D: closed db index /var/lib/rpm/Providename
>>>>>>>> D: closed db index /var/lib/rpm/Name
>>>>>>>> D: closed db index /var/lib/rpm/Packages
>>>>>>>> D: closed db environment /var/lib/rpm
>>>>>>>>
>>>>>>>>
>>>>>>>> 2015-01-27 10:53 GMT+00:00 Alexander Innes <senni at necurity.co.uk>:
>>>>>>>>
>>>>>>>>> Schedual the action then on the server do rhn-chck -vvv. It will
>>>>>>>>> tell you what the client's doing, my gut feeling is either SSL errors OR
>>>>>>>>> permsions (rhn-actions-control is it?)
>>>>>>>>>
>>>>>>>>> On 26 January 2015 at 23:16, Chris Swingler <
>>>>>>>>> chris at chrisswingler.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hey Spacewalk-list!
>>>>>>>>>>
>>>>>>>>>> I have quite a few systems (I haven't dug into seeing if there is
>>>>>>>>>> any consistency in CentOS versions or the RHN tools) that just plain do not
>>>>>>>>>> seem to get tasks back from the Spacewalk server. We're running Spacewalk
>>>>>>>>>> 2.2 on CentOS release 6.5 (Final).
>>>>>>>>>>
>>>>>>>>>> I can create a simple task in Spacewalk, like just print the
>>>>>>>>>> output of "date", and an rhn_check never seems to execute it. It shows up
>>>>>>>>>> under Events > Pending, the SystemIDs match, but it never moves.
>>>>>>>>>>
>>>>>>>>>> Running a packet capture to watch the transaction, it looks like
>>>>>>>>>> Spacewalk itself isn't even sending the task to the remote system.
>>>>>>>>>> Spacewalk happily replies with an HTTP 200 and an empty payload.
>>>>>>>>>>
>>>>>>>>>> HTTP/1.1 200 OK
>>>>>>>>>> Date: Mon, 26 Jan 2015 22:47:55 GMT
>>>>>>>>>> Server: Apache
>>>>>>>>>> Content-Length: 126
>>>>>>>>>> X-RHN-Server-Capability: registration.finish_message(1)=1
>>>>>>>>>> X-RHN-Server-Capability: registration.remaining_subscriptions(1)=1
>>>>>>>>>> X-RHN-Server-Capability: abrt(1)=1
>>>>>>>>>> X-RHN-Server-Capability: registration.update_contact_info(1)=1
>>>>>>>>>> X-RHN-Server-Capability: staging_content(1)=1
>>>>>>>>>> X-RHN-Server-Capability: applet.has_base_channel(1)=1
>>>>>>>>>> X-RHN-Server-Capability: registration.smbios(1)=1
>>>>>>>>>> X-RHN-Server-Capability: registration.extended_update_support(1)=1
>>>>>>>>>> X-RHN-Server-Capability: rhncfg.filetype.directory(1)=1
>>>>>>>>>> X-RHN-Server-Capability: registration.update_systemid(1)=1
>>>>>>>>>> X-RHN-Server-Capability: registration.register_osad(1)=1
>>>>>>>>>> X-RHN-Server-Capability: registration.delta_packages(1)=1
>>>>>>>>>> X-RHN-Server-Capability: cpu_sockets(1)=1
>>>>>>>>>> X-RHN-Server-Capability: ipv6(1)=1
>>>>>>>>>> X-RHN-Server-Capability: rhncfg.content.base64_decode(1)=1
>>>>>>>>>> X-RHN-Server-Capability: xmlrpc.packages.extended_profile(1-2)=1
>>>>>>>>>> X-RHN-Server-Capability: xmlrpc.errata.patch_names(1)=1
>>>>>>>>>> X-RHN-Server-Capability: xmlrpc.packages.checksums(1)=1
>>>>>>>>>> X-RHN-Server-Capability: xmlrpc.login.extra_data(1)=1
>>>>>>>>>> X-RHN-Proxy-Version: 0
>>>>>>>>>> X-Transport-Info: Extended Capabilities Transport (C) Red Hat, Inc (version 2.5.72-1.el6)
>>>>>>>>>> X-RHN-Client-Version: 1
>>>>>>>>>> Connection: close
>>>>>>>>>> Content-Type: text/xml
>>>>>>>>>>
>>>>>>>>>> <?xml version='1.0'?>
>>>>>>>>>> <methodResponse>
>>>>>>>>>> <params>
>>>>>>>>>> <param>
>>>>>>>>>> <value><string></string></value>
>>>>>>>>>> </param>
>>>>>>>>>> </params>
>>>>>>>>>> </methodResponse>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Full transaction at
>>>>>>>>>>
>>>>>>>>>> https://gist.github.com/cswingler/f718abcb9ce290adec29
>>>>>>>>>>
>>>>>>>>>> rhn_server_xmlrpc.log simply shows:
>>>>>>>>>>
>>>>>>>>>> 2015/01/26 16:52:07 -05:00 30997 172.29.7.14:
>>>>>>>>>> xmlrpc/queue.get(1000015056, 2, 'checkins enabled')
>>>>>>>>>> 2015/01/26 16:52:07 -05:00 30998 172.29.7.14:
>>>>>>>>>> xmlrpc/up2date.listChannels(1000015056,)
>>>>>>>>>> 2015/01/26 16:52:08 -05:00 30995 172.29.7.14:
>>>>>>>>>> xmlrpc/registration.welcome_message('lang: None',)
>>>>>>>>>>
>>>>>>>>>> Any basic things I should be checking? I've tried updating the
>>>>>>>>>> spacewalk/rhn tools on the agent, restarting Spacewalk itself, and nothing
>>>>>>>>>> seems to make a change. This issue, as far as I can recall, did not seem to
>>>>>>>>>> appear until after we upgraded to 2.2.
>>>>>>>>>>
>>>>>>>>>> Some systems seem just fine, and I can't seem to find a
>>>>>>>>>> consistent reason why some succeed and others fail.
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Spacewalk-list mailing list
>>>>>>>>>> Spacewalk-list at redhat.com
>>>>>>>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Spacewalk-list mailing list
>>>>>>>>> Spacewalk-list at redhat.com
>>>>>>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Paulo Silva <paulojjs at gmail.com>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Spacewalk-list mailing list
>>>>>>>> Spacewalk-list at redhat.com
>>>>>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Spacewalk-list mailing list
>>>>>>> Spacewalk-list at redhat.com
>>>>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Paulo Silva <paulojjs at gmail.com>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Spacewalk-list mailing list
>>>>>> Spacewalk-list at redhat.com
>>>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Spacewalk-list mailing list
>>>>> Spacewalk-list at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Spacewalk-list mailing list
>>>>> Spacewalk-list at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Paulo Silva <paulojjs at gmail.com>
>>>>
>>>
>>>
>>>
>>> --
>>> Paulo Silva <paulojjs at gmail.com>
>>>
>>> _______________________________________________
>>> Spacewalk-list mailing list
>>> Spacewalk-list at redhat.com
>>> https://www.redhat.com/mailman/listinfo/spacewalk-list
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/spacewalk-list/attachments/20150127/5a48f12d/attachment.htm>
More information about the Spacewalk-list
mailing list