[Pulp-list] pulp3 tasks 500 error

Mon Oct 28 20:03:13 UTC 2019

Here is the traceback:

Oct 28 15:58:45 ipaddr gunicorn[165752]: 127.0.0.1 - admin [28/Oct/2019:19:58:45 +0000] "GET /pulp/api/v3/status/ HTTP/1.0" 200 1366 "-" "HTTPie/0.9.4"
Oct 28 15:58:50 ipaddr gunicorn[165752]: 127.0.0.1 - admin [28/Oct/2019:19:58:50 +0000] "GET /pulp/api/v3/status/ HTTP/1.0" 200 1366 "-" "HTTPie/0.9.4"
Oct 28 15:58:55 ipaddr gunicorn[165752]: pulp: django.request:WARNING: Not Found: /pulp/api/v3/tasks/rpm/rpm/
Oct 28 15:58:55 ipaddr gunicorn[165752]: 127.0.0.1 - admin [28/Oct/2019:19:58:55 +0000] "GET /pulp/api/v3/tasks/rpm/rpm/ HTTP/1.1" 404 77 "-" "HTTPie/0.9.4"
Oct 28 15:59:01 ipaddr systemd[1]: Started Session 50160 of user root.
Oct 28 15:59:08 ipaddr gunicorn[165752]: pulp: django.request:ERROR: Internal Server Error: /pulp/api/v3/tasks/
Oct 28 15:59:08 ipaddr gunicorn[165752]: Traceback (most recent call last):
Oct 28 15:59:08 ipaddr gunicorn[165752]: File "/opt/utils/venv/pulp/3.6.5/lib64/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
Oct 28 15:59:08 ipaddr gunicorn[165752]: response = get_response(request)
Oct 28 15:59:08 ipaddr gunicorn[165752]: File "/opt/utils/venv/pulp/3.6.5/lib64/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
Oct 28 15:59:08 ipaddr gunicorn[165752]: response = self.process_exception_by_middleware(e, request)
Oct 28 15:59:08 ipaddr gunicorn[165752]: File "/opt/utils/venv/pulp/3.6.5/lib64/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
Oct 28 15:59:08 ipaddr gunicorn[165752]: response = wrapped_callback(request, *callback_args, **callback_kwargs)
Oct 28 15:59:08 ipaddr gunicorn[165752]: File "/opt/utils/venv/pulp/3.6.5/lib64/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_vie
Oct 28 15:59:08 ipaddr gunicorn[165752]: return view_func(*args, **kwargs)
Oct 28 15:59:08 ipaddr gunicorn[165752]: File "/opt/utils/venv/pulp/3.6.5/lib64/python3.6/site-packages/rest_framework/viewsets.py", line 114, in view
Oct 28 15:59:08 ipaddr gunicorn[165752]: return self.dispatch(request, *args, **kwargs)
Oct 28 15:59:08 ipaddr gunicorn[165752]: File "/opt/utils/venv/pulp/3.6.5/lib64/python3.6/site-packages/rest_framework/views.py", line 505, in dispatch
Oct 28 15:59:08 ipaddr gunicorn[165752]: response = self.handle_exception(exc)
Oct 28 15:59:08 ipaddr gunicorn[165752]: File "/opt/utils/venv/pulp/3.6.5/lib64/python3.6/site-packages/rest_framework/views.py", line 465, in handle_exception
Oct 28 15:59:08 ipaddr gunicorn[165752]: self.raise_uncaught_exception(exc)
Oct 28 15:59:08 ipaddr gunicorn[165752]: File "/opt/utils/venv/pulp/3.6.5/lib64/python3.6/site-packages/rest_framework/views.py", line 476, in raise_uncaught_exc
Oct 28 15:59:08 ipaddr gunicorn[165752]: raise exc
Oct 28 15:59:08 ipaddr gunicorn[165752]: File "/opt/utils/venv/pulp/3.6.5/lib64/python3.6/site-packages/rest_framework/views.py", line 502, in dispatch
Oct 28 15:59:08 ipaddr gunicorn[165752]: response = handler(request, *args, **kwargs)
Oct 28 15:59:08 ipaddr gunicorn[165752]: File "/opt/utils/venv/pulp/3.6.5/lib64/python3.6/site-packages/rest_framework/mixins.py", line 43, in list
Oct 28 15:59:08 ipaddr gunicorn[165752]: return self.get_paginated_response(serializer.data)
Oct 28 15:59:08 ipaddr gunicorn[165752]: File "/opt/utils/venv/pulp/3.6.5/lib64/python3.6/site-packages/rest_framework/serializers.py", line 757, in data
Oct 28 15:59:08 ipaddr gunicorn[165752]: ret = super().data
Oct 28 15:59:08 ipaddr gunicorn[165752]: File "/opt/utils/venv/pulp/3.6.5/lib64/python3.6/site-packages/rest_framework/serializers.py", line 261, in data
Oct 28 15:59:08 ipaddr gunicorn[165752]: self._data = self.to_representation(self.instance)
Oct 28 15:59:08 ipaddr gunicorn[165752]: File "/opt/utils/venv/pulp/3.6.5/lib64/python3.6/site-packages/rest_framework/serializers.py", line 675, in to_represent
Oct 28 15:59:08 ipaddr gunicorn[165752]: self.child.to_representation(item) for item in iterable
Oct 28 15:59:08 ipaddr gunicorn[165752]: File "/opt/utils/venv/pulp/3.6.5/lib64/python3.6/site-packages/rest_framework/serializers.py", line 675, in <listcomp>
Oct 28 15:59:08 ipaddr gunicorn[165752]: self.child.to_representation(item) for item in iterable
Oct 28 15:59:08 ipaddr gunicorn[165752]: File "/opt/utils/venv/pulp/3.6.5/lib64/python3.6/site-packages/rest_framework/serializers.py", line 526, in to_represent
Oct 28 15:59:08 ipaddr gunicorn[165752]: ret[field.field_name] = field.to_representation(attribute)
Oct 28 15:59:08 ipaddr gunicorn[165752]: File "/opt/utils/venv/pulp/3.6.5/lib64/python3.6/site-packages/rest_framework/fields.py", line 1721, in to_representatio
Oct 28 15:59:08 ipaddr gunicorn[165752]: for key, val in value.items()
Oct 28 15:59:08 ipaddr gunicorn[165752]: AttributeError: 'str' object has no attribute 'items'
Oct 28 15:59:08 ipaddr gunicorn[165752]: 127.0.0.1 - admin [28/Oct/2019:19:59:08 +0000] "GET /pulp/api/v3/tasks/ HTTP/1.0" 500 27 "-" "HTTPie/0.9.4"

From: dkliban at redhat.com At: 10/28/19 15:37:34To:  Bin Li (BLOOMBERG/ 120 PARK ) 
Cc:  pulp-list at redhat.com
Subject: Re: [Pulp-list] pulp3 tasks 500 error

Do you see anything in journalctl when the 500 response is generated? I would expect a traceback in the pulp-api logs. 

On Mon, Oct 28, 2019 at 3:16 PM Bin Li (BLOOMBERG/ 120 PARK) <bli111 at bloomberg.net> wrote:

During a test to sync about 100 repositories, we suddenly got Error 500 from querying task status after first 20 sync tasks completed. Tried to restarted all pulp daemons and still getting 500 errors. Both port 24816 and 24817 are still listening and I am able to query postgres db with psql. How should I troubleshot this issue? Please advise.

Thanks_______________________________________________
Pulp-list mailing list
Pulp-list at redhat.com
https://www.redhat.com/mailman/listinfo/pulp-list

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20191028/9f16f62a/attachment.htm>