[Pulp-list] Pulp 2.4 RHEL 7 child node sync fails
Cristian Falcas
cristi.falcas at gmail.com
Mon Sep 15 11:33:13 UTC 2014
I think it's something different, so I opened 1141743 for this.
I also test it on el6 and I got the same error.
On Mon, Sep 15, 2014 at 1:16 PM, johan petersson <j.petersson at gmail.com>
wrote:
> This problem might be related to this bug:
> https://bugzilla.redhat.com/show_bug.cgi?id=1129584
>
> On Mon, Sep 15, 2014 at 11:25 AM, Cristian Falcas <cristi.falcas at gmail.com
> > wrote:
>
>> We get the same error when trying to sync or delete a puppet local repo
>> (also el7):
>>
>> 2014-09-14 16:40:27,016 - INFO - DELETE request to
>> /pulp/api/v2/repositories/puppet-builds/ with parameters None
>> 2014-09-14 16:40:27,017 - INFO - Response status : 500
>>
>> 2014-09-14 16:40:27,017 - INFO - Response body :
>> {
>> "exception": [
>> "AuthenticationFailure: connection-forced: Authentication
>> failed(320)\n"
>> ],
>> "traceback": [
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/middleware/exception.py\",
>> line 44, in __call__\n return self.app(environ, start_response)\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/middleware/postponed.py\",
>> line 42, in __call__\n return self.app(environ, start_response)\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 279, in wsgi\n result = self.handle_with_processors()\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 29, in _handle_with_processors\n return process(self.processors)\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 26, in process\n return p(lambda: process(processors))\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 566, in processor\n return handler()\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 26, in <lambda>\n return p(lambda: process(processors))\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 26, in process\n return p(lambda: process(processors))\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 581, in processor\n result = handler()\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 26, in <lambda>\n return p(lambda: process(processors))\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 28, in process\n return self.handle()\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 230, in handle\n return self._delegate(fn, self.fvars, args)\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 422, in _delegate\n return f()\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 430, in <lambda>\n f = lambda: self._delegate_sub_application(pat,
>> what)\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 455, in _delegate_sub_application\n return
>> app.handle_with_processors()\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 29, in _handle_with_processors\n return process(self.processors)\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 26, in process\n return p(lambda: process(processors))\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 566, in processor\n return handler()\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 26, in <lambda>\n return p(lambda: process(processors))\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 26, in process\n return p(lambda: process(processors))\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 581, in processor\n result = handler()\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 26, in <lambda>\n return p(lambda: process(processors))\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 28, in process\n return self.handle()\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 230, in handle\n return self._delegate(fn, self.fvars, args)\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 420, in _delegate\n return handle_class(cls)\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 396, in handle_class\n return tocall(*args)\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/controllers/decorators.py\",
>> line 227, in _auth_decorator\n value = method(self, *args, **kwargs)\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/controllers/repositories.py\",
>> line 257, in DELETE\n [repo_id], tags=task_tags)\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 234,
>> in apply_async_with_reservation\n queue =
>> _reserve_resource.apply_async((resource_id,),
>> queue=RESOURCE_MANAGER_QUEUE).get()\n",
>> " File \"/usr/lib/python2.7/site-packages/celery/app/task.py\", line
>> 555, in apply_async\n **dict(self._get_exec_options(), **options)\n",
>> " File \"/usr/lib/python2.7/site-packages/celery/app/base.py\", line
>> 351, in send_task\n reply_to=reply_to or self.oid, **options\n",
>> " File \"/usr/lib/python2.7/site-packages/celery/app/amqp.py\", line
>> 305, in publish_task\n **kwargs\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/messaging.py\", line
>> 168, in publish\n routing_key, mandatory, immediate, exchange,
>> declare)\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/connection.py\",
>> line 440, in _ensured\n return fun(*args, **kwargs)\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/messaging.py\", line
>> 173, in _publish\n channel = self.channel\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/messaging.py\", line
>> 190, in _get_channel\n channel = self._channel = channel()\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/utils/__init__.py\",
>> line 422, in __call__\n value = self.__value__ = self.__contract__()\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/messaging.py\", line
>> 205, in <lambda>\n channel = ChannelPromise(lambda:
>> connection.default_channel)\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/connection.py\",
>> line 773, in default_channel\n self.connection\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/connection.py\",
>> line 758, in connection\n self._connection =
>> self._establish_connection()\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/connection.py\",
>> line 713, in _establish_connection\n conn =
>> self.transport.establish_connection()\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/transport/qpid.py\",
>> line 1673, in establish_connection\n conn = self.Connection(**opts)\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/transport/qpid.py\",
>> line 1365, in __init__\n self._qpid_conn =
>> establish(**self.connection_options)\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py\", line 68,
>> in establish\n conn.open(timeout=timeout)\n",
>> " File \"<string>\", line 6, in open\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py\", line 268,
>> in open\n self.attach(timeout=timeout)\n",
>> " File \"<string>\", line 6, in attach\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py\", line 286,
>> in attach\n if not self._ewait(lambda: self._transport_connected and not
>> self._unlinked(), timeout=timeout):\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py\", line 219,
>> in _ewait\n self.check_error()\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py\", line 212,
>> in check_error\n raise e\n"
>> ],
>> "_href": "/pulp/api/v2/repositories/puppet-builds/",
>> "error_message": "Unhandled Exception",
>> "http_request_method": "DELETE",
>> "http_status": 500
>> }
>>
>>
>>
>> and from sync:
>>
>> 2014-09-14 16:32:00,422 - INFO - POST request to
>> /pulp/api/v2/tasks/search/ with parameters {"criteria": {"filters":
>> {"state": {"$nin": ["finished", "error", "canceled", "skipped"]}, "tags":
>> {"$all": ["pulp:repository:testes", "pulp:action:sync"]}}}}
>> 2014-09-14 16:32:00,422 - INFO - Response status : 200
>>
>> 2014-09-14 16:32:00,422 - INFO - Response body :
>> []
>>
>> 2014-09-14 16:32:00,601 - INFO - POST request to
>> /pulp/api/v2/repositories/testes/actions/sync/ with parameters
>> {"override_config": null}
>> 2014-09-14 16:32:00,601 - INFO - Response status : 500
>>
>> 2014-09-14 16:32:00,602 - INFO - Response body :
>> {
>> "exception": [
>> "AuthenticationFailure: connection-forced: Authentication
>> failed(320)\n"
>> ],
>> "traceback": [
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/middleware/exception.py\",
>> line 44, in __call__\n return self.app(environ, start_response)\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/middleware/postponed.py\",
>> line 42, in __call__\n return self.app(environ, start_response)\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 279, in wsgi\n result = self.handle_with_processors()\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 29, in _handle_with_processors\n return process(self.processors)\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 26, in process\n return p(lambda: process(processors))\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 566, in processor\n return handler()\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 26, in <lambda>\n return p(lambda: process(processors))\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 26, in process\n return p(lambda: process(processors))\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 581, in processor\n result = handler()\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 26, in <lambda>\n return p(lambda: process(processors))\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 28, in process\n return self.handle()\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 230, in handle\n return self._delegate(fn, self.fvars, args)\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 422, in _delegate\n return f()\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 430, in <lambda>\n f = lambda: self._delegate_sub_application(pat,
>> what)\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 455, in _delegate_sub_application\n return
>> app.handle_with_processors()\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 29, in _handle_with_processors\n return process(self.processors)\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 26, in process\n return p(lambda: process(processors))\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 566, in processor\n return handler()\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 26, in <lambda>\n return p(lambda: process(processors))\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 26, in process\n return p(lambda: process(processors))\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 581, in processor\n result = handler()\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 26, in <lambda>\n return p(lambda: process(processors))\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/application.py\",
>> line 28, in process\n return self.handle()\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 230, in handle\n return self._delegate(fn, self.fvars, args)\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 420, in _delegate\n return handle_class(cls)\n",
>> " File \"/usr/lib/python2.7/site-packages/web/application.py\", line
>> 396, in handle_class\n return tocall(*args)\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/controllers/decorators.py\",
>> line 227, in _auth_decorator\n value = method(self, *args, **kwargs)\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/webservices/controllers/repositories.py\",
>> line 710, in POST\n repo_id, [repo_id, overrides], {},
>> tags=task_tags)\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 234,
>> in apply_async_with_reservation\n queue =
>> _reserve_resource.apply_async((resource_id,),
>> queue=RESOURCE_MANAGER_QUEUE).get()\n",
>> " File \"/usr/lib/python2.7/site-packages/celery/app/task.py\", line
>> 555, in apply_async\n **dict(self._get_exec_options(), **options)\n",
>> " File \"/usr/lib/python2.7/site-packages/celery/app/base.py\", line
>> 351, in send_task\n reply_to=reply_to or self.oid, **options\n",
>> " File \"/usr/lib/python2.7/site-packages/celery/app/amqp.py\", line
>> 305, in publish_task\n **kwargs\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/messaging.py\", line
>> 168, in publish\n routing_key, mandatory, immediate, exchange,
>> declare)\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/connection.py\",
>> line 440, in _ensured\n return fun(*args, **kwargs)\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/messaging.py\", line
>> 173, in _publish\n channel = self.channel\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/messaging.py\", line
>> 190, in _get_channel\n channel = self._channel = channel()\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/utils/__init__.py\",
>> line 422, in __call__\n value = self.__value__ = self.__contract__()\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/messaging.py\", line
>> 205, in <lambda>\n channel = ChannelPromise(lambda:
>> connection.default_channel)\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/connection.py\",
>> line 773, in default_channel\n self.connection\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/connection.py\",
>> line 758, in connection\n self._connection =
>> self._establish_connection()\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/connection.py\",
>> line 713, in _establish_connection\n conn =
>> self.transport.establish_connection()\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/transport/qpid.py\",
>> line 1673, in establish_connection\n conn = self.Connection(**opts)\n",
>> " File \"/usr/lib/python2.7/site-packages/kombu/transport/qpid.py\",
>> line 1365, in __init__\n self._qpid_conn =
>> establish(**self.connection_options)\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py\", line 68,
>> in establish\n conn.open(timeout=timeout)\n",
>> " File \"<string>\", line 6, in open\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py\", line 268,
>> in open\n self.attach(timeout=timeout)\n",
>> " File \"<string>\", line 6, in attach\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py\", line 286,
>> in attach\n if not self._ewait(lambda: self._transport_connected and not
>> self._unlinked(), timeout=timeout):\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py\", line 219,
>> in _ewait\n self.check_error()\n",
>> " File
>> \"/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py\", line 212,
>> in check_error\n raise e\n"
>> ],
>> "_href": "/pulp/api/v2/repositories/testes/actions/sync/",
>> "error_message": "Unhandled Exception",
>> "http_request_method": "POST",
>> "http_status": 500
>> }
>>
>>
>>
>>
>>
>> On Mon, Sep 15, 2014 at 12:11 PM, johan petersson <j.petersson at gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> I can't seem to get sync to a child node to work.
>>> It is a new fresh installation - no upgrade.
>>> I followed all the instructions for Pulp Server and Consumer
>>> installation and that part works.
>>> However when using the instructions here for the node configuration:
>>> http://pulp-user-guide.readthedocs.org/en/latest/nodes.html
>>> http://pulp-user-guide.readthedocs.org/en/latest/nodes.html
>>>
>>> Node synchronization fails with the following error message: -01: An
>>> unexpected error occurred. repo_id=None
>>>
>>> I have set oauth in both server.conf and nodes.conf correctly, using
>>> default certs everywhere and have set verify_ssl: False.
>>>
>>> I have even tried new installation on new Vm's but get the same result.
>>>
>>> Here is some info from the log:
>>>
>>> Sep 15 10:42:05 repo pulp:
>>> pulp.server.webservices.middleware.exception:ERROR: File
>>> "/usr/lib/python2.7/site-packages/pulp/server/webservices/controllers/decorators.py",
>>> line 192, in _auth_decorator
>>>
>>> Sep 15 10:42:05 repo pulp:
>>> pulp.server.webservices.middleware.exception:ERROR: userid, is_consumer
>>> = authenticate_user()
>>>
>>> Sep 15 10:42:05 repo pulp:
>>> pulp.server.webservices.middleware.exception:ERROR: File
>>> "/usr/lib/python2.7/site-packages/pulp/server/webservices/controllers/decorators.py",
>>> line 111, in oauth_authentication
>>>
>>> Sep 15 10:42:05 repo pulp:
>>> pulp.server.webservices.middleware.exception:ERROR: raise
>>> AuthenticationFailed(auth_utils.CODE_INVALID_SSL_CERT)
>>>
>>> Sep 15 10:42:05 repo pulp:
>>> pulp.server.webservices.middleware.exception:ERROR: AuthenticationFailed:
>>> Pulp exception occurred: AuthenticationFailed
>>>
>>> Sep 15 10:42:05 repo goferd: [ERROR][worker-0]
>>> pulp_node.handlers.strategies:129 - synchronization failed
>>>
>>> Sep 15 10:42:05 repo goferd: [ERROR][worker-0]
>>> pulp_node.handlers.strategies:129 - Traceback (most recent call last): File
>>> "/usr/lib/python2.7/site-packages/pulp_node/handlers/strategies.py", line
>>> 116, in synchronize validator.validate(request.bindings) File
>>> "/usr/lib/python2.7/site-packages/pulp_node/handlers/validation.py", line
>>> 54, in validate self.report.errors.extend(self._validate_plugins(bindings))
>>> File "/usr/lib/python2.7/site-packages/pulp_node/handlers/validation.py",
>>> line 70, in _validate_plugins child = ChildServer() File
>>> "/usr/lib/python2.7/site-packages/pulp_node/handlers/validation.py", line
>>> 88, in __init__ self.importers = self._importers() File
>>> "/usr/lib/python2.7/site-packages/pulp_node/handlers/validation.py", line
>>> 99, in _importers http = bindings.server_info.get_importers() File
>>> "/usr/lib/python2.7/site-packages/pulp/bindings/server_info.py", line 43,
>>> in get_importers return self.server.GET(path) File
>>> "/usr/lib/python2.7/site-packages/pulp/bindings/server.py", line 93, in GET
>>> return self._request('GET', path, queries) File
>>> "/usr/lib/python2.7/site-packages/pulp/bindings/server.py", line 151, in
>>> _request self._handle_exceptions(response_code, response_body) File
>>> "/usr/lib/python2.7/site-packages/pulp/bindings/server.py", line 192, in
>>> _handle_exceptions raise code_class_mappings[response_code](response_body)
>>> PermissionsException: RequestException: GET request on
>>> /pulp/api/v2/plugins/importers/ failed with 401 - Pulp exception occurred:
>>> AuthenticationFailed
>>>
>>> Sep 15 10:42:05 repo goferd: [ERROR][worker-0]
>>> pulp_node.handlers.handler:111 - An unexpected error occurred. repo_id=None
>>>
>>> Sep 15 10:42:05 repo goferd: [INFO][worker-0] gofer.agent.rmi:149 -
>>> sn=7b3315f2-ec18-4ced-be8b-2d714ae0b73b processed in: 114 (ms)
>>>
>>>
>>> It seems clear that it is a certificate error but i use verify_ssl:
>>> False
>>>
>>>
>>> I have also tried: pulp-consumer update --keys and restarted httpd and
>>> goferd with same result.
>>>
>>>
>>> I am now out of ideas, anyone here have any idea how to fix this problem?
>>>
>>>
>>> Thank you for any help,
>>>
>>>
>>> Johan
>>>
>>> _______________________________________________
>>> Pulp-list mailing list
>>> Pulp-list at redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-list
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20140915/bdfb0f0d/attachment.htm>
More information about the Pulp-list
mailing list