[Pulp-list] pypi sync no longer working after pypi.org migration with pulp 2.14 and pulp-python-plugins 2.0.2-1

Wed Jul 25 14:14:50 UTC 2018

Sure: https://pulp.plan.io/issues/3882 <https://pulp.plan.io/issues/3882>

Dustin McNabb


> On Jul 25, 2018, at 6:55 AM, Bihan Zhang <bizhang at redhat.com> wrote:
> 
> Dustin,
> Would you mind opening a pulp issue with a description of the issue and the patch? https://pulp.plan.io/issues/new <https://pulp.plan.io/issues/new> 
> I'm not too familiar with the tasking system, and it would be good for the issue to go through the regular triage process.
> 
> 
> Cheers,
> Bihan
> 
> On Wed, Jul 25, 2018 at 8:18 AM, Dustin McNabb <dustin.mcnabb at gmail.com <mailto:dustin.mcnabb at gmail.com>> wrote:
> Bihan -
> 
> Thanks for the fast response. However, I’ve upgraded my lab from pulp 2.14 to this beta and it introduces a new problem by upgrading to celery 4.0.2-4. Specifically, we now have this issue https://github.com/celery/celery/issues/3620 <https://github.com/celery/celery/issues/3620>. It was fixed in celery 4.1. You need multiple tasks in the queue upon initial startup to reproduce this issue. You can also see more discussion about this issue here: https://github.com/celery/celery/pull/3752 <https://github.com/celery/celery/pull/3752> and here https://github.com/celery/celery/issues/4109#issuecomment-314367686 <https://github.com/celery/celery/issues/4109#issuecomment-314367686>. I implemented this fix in my lab environment and it resolved my issue.
> 
> # Problem with pulp_resource_manager service
> Jul 25 06:10:03 pulp-lab pulp[17698]: celery.worker:CRITICAL: (17698-58784)   File "/usr/lib64/python2.7/site-packages/billiard/pool.py", line 1486, in apply_async
> Jul 25 06:10:03 pulp-lab  pulp[17698]: celery.worker:CRITICAL: (17698-58784)     self._quick_put((TASK, (result._job, None, func, args, kwds)))
> Jul 25 06:10:03 pulp-lab pulp[17698]: celery.worker:CRITICAL: (17698-58784) TypeError: 'NoneType' object is not callable
> 
> # Here is the fix that was ultimately implemented which moves the consumption step below the object registration in worker/loops.py.
> -    consumer.consume()
> -    obj.on_ready()
>      obj.controller.register_with_event_loop(hub)
>      obj.register_with_event_loop(hub)
> +    consumer.consume()
> +    obj.on_ready()
> 
> 
> Thanks
> Dustin McNabb
> 
> 
> 
> 
>> On Jul 24, 2018, at 2:23 PM, Bihan Zhang <bizhang at redhat.com <mailto:bizhang at redhat.com>> wrote:
>> 
>> Dustin,
>> 
>> These fixes were implemented[0] and were just release in pulp 2.16.4 beta and pulp_python 2.0.3 beta today.
>> Here's our release schedule in case you want to keep an eye out on when this is going to go GA: https://pulp.plan.io/projects/pulp/wiki/2164_Release_Status <https://pulp.plan.io/projects/pulp/wiki/2164_Release_Status>
>> 
>> 
>> [0] https://pulp.plan.io/issues/3578 <https://pulp.plan.io/issues/3578>
>> 
>> On Tue, Jul 24, 2018 at 4:50 PM, Dustin McNabb <dustin.mcnabb at gmail.com <mailto:dustin.mcnabb at gmail.com>> wrote:
>> I have pulp-server 2.14.3-1 and pulp-python-plugins 2.0.2-1 and I can no longer sync form python.pypi.org <http://python.pypi.org/>. I realize they switched to pypi.org <http://pypi.org/> and files.pythonhosted.org <http://files.pythonhosted.org/>, but the old site still redirects requests. However, the URLs where pulp-python-plugins expects to find packages no longer exist. Also, the pulp-python packages have not been updated since 2017. Does this mean that we don’t have support for the new pypi.org <http://pypi.org/> warehouse using pulp2? Or is there a legacy feed URL I can use?
>> 
>> Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://pypi.org/simple/pypi/tinydb/json/ <https://pypi.org/simple/pypi/tinydb/json/> failed with code 404: Not Found
>> Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (37): pypi.org <http://pypi.org/>
>> Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://pypi.org/simple/pypi/setuptools/json/ <https://pypi.org/simple/pypi/setuptools/json/> failed with code 404: Not Found
>> Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (38): pypi.org <http://pypi.org/>
>> Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://pypi.org/simple/pypi/urllib3/json/ <https://pypi.org/simple/pypi/urllib3/json/> failed with code 404: Not Found
>> Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (39): pypi.org <http://pypi.org/>
>> Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://pypi.org/simple/pypi/chardet/json/ <https://pypi.org/simple/pypi/chardet/json/> failed with code 404: Not Found
>> Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (40): pypi.org <http://pypi.org/>
>> Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://pypi.org/simple/pypi/passlib/json/ <https://pypi.org/simple/pypi/passlib/json/> failed with code 404: Not Found
>> Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (41): pypi.org <http://pypi.org/>
>> Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://pypi.org/simple/pypi/setuptools/json/ <https://pypi.org/simple/pypi/setuptools/json/> failed with code 404: Not Found
>> Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (42): pypi.org <http://pypi.org/>
>> Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://pypi.org/simple/pypi/gunicorn/json/ <https://pypi.org/simple/pypi/gunicorn/json/> failed with code 404: Not Found
>> Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://pypi.org/simple/pypi/idna/json/ <https://pypi.org/simple/pypi/idna/json/> failed with code 404: Not Found
>> 
>> Thanks
>> 
>> Dustin McNabb
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> Pulp-list mailing list
>> Pulp-list at redhat.com <mailto:Pulp-list at redhat.com>
>> https://www.redhat.com/mailman/listinfo/pulp-list <https://www.redhat.com/mailman/listinfo/pulp-list>
>> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20180725/e1c2b230/attachment.htm>