<div dir="ltr">Dustin, <div><br></div><div>Thanks for the heads up. We're looking into it now and will have a fix soon.<br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><br></div><div>David<br></div></div></div></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, Jul 25, 2018 at 1:29 PM Dustin McNabb <<a href="mailto:dustin.mcnabb@gmail.com">dustin.mcnabb@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space">Apparently this essential <a href="http://pypi.org" target="_blank">pypi.org</a> fix at the github link below was left out of the pulp-python-2.0.3-0.1 release for the pulp 2.16.4 beta. It is essential for <a href="http://pypi.org" target="_blank">pypi.org</a> support as path is no longer an option and the url formation logic in models.py no longer applies. I have opened another issue at <a href="https://pulp.plan.io/issues/3884" target="_blank">https://pulp.plan.io/issues/3884</a> <div><br></div><div><a href="https://github.com/pulp/pulp_python/commit/79b46411a049e63fc24ba97a60b51fe0574b268c" target="_blank">https://github.com/pulp/pulp_python/commit/79b46411a049e63fc24ba97a60b51fe0574b268c</a></div><div><br><div>
<span class="m_-4997864212792679721Apple-style-span" style="border-collapse:separate;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-alternates:normal;font-variant-east-asian:normal;line-height:normal;border-spacing:0px"><div style="word-wrap:break-word"><div>Dustin McNabb</div></div></span><br class="m_-4997864212792679721Apple-interchange-newline"><br class="m_-4997864212792679721Apple-interchange-newline">
</div>
<div><br><blockquote type="cite"><div>On Jul 25, 2018, at 7:32 AM, Brian Bouterse <<a href="mailto:bbouters@redhat.com" target="_blank">bbouters@redhat.com</a>> wrote:</div><br class="m_-4997864212792679721Apple-interchange-newline"><div><div dir="ltr">Thank you for investigating and reporting this. We are going to try to reproduce and patch the version of Celery we distribute with Pulp.<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jul 25, 2018 at 10:14 AM, Dustin McNabb <span dir="ltr"><<a href="mailto:dustin.mcnabb@gmail.com" target="_blank">dustin.mcnabb@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space">Sure: <a href="https://pulp.plan.io/issues/3882" target="_blank">https://pulp.plan.io/issues/3882</a><div><span class="m_-4997864212792679721HOEnZb"><font color="#888888"><br><div>
<span class="m_-4997864212792679721m_7540986556227809640Apple-style-span" style="border-collapse:separate;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-alternates:normal;font-variant-east-asian:normal;line-height:normal;border-spacing:0px"><div style="word-wrap:break-word"><div>Dustin McNabb</div><div><br></div></div></span><br class="m_-4997864212792679721m_7540986556227809640Apple-interchange-newline"><br class="m_-4997864212792679721m_7540986556227809640Apple-interchange-newline">
</div></font></span><div><div class="m_-4997864212792679721h5">
<div><br><blockquote type="cite"><div>On Jul 25, 2018, at 6:55 AM, Bihan Zhang <<a href="mailto:bizhang@redhat.com" target="_blank">bizhang@redhat.com</a>> wrote:</div><br class="m_-4997864212792679721m_7540986556227809640Apple-interchange-newline"><div><div dir="ltr">Dustin,<div>Would you mind opening a pulp issue with a description of the issue and the patch? <span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><a href="https://pulp.plan.io/issues/new" target="_blank">https://pulp.plan.io/issues/new</a><span> </span></span></div><div>I'm not too familiar with the tasking system, and it would be good for the issue to go through the regular triage process.</div><div><br></div><div><br></div><div>Cheers,</div><div>Bihan</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Jul 25, 2018 at 8:18 AM, Dustin McNabb <span dir="ltr"><<a href="mailto:dustin.mcnabb@gmail.com" target="_blank">dustin.mcnabb@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space">Bihan -<div><br></div><div>Thanks for the fast response. However, I’ve upgraded my lab from pulp 2.14 to this beta and it introduces a new problem by upgrading to celery 4.0.2-4. Specifically, we now have this issue <a href="https://github.com/celery/celery/issues/3620" target="_blank">https://github.com/celery/celery/issues/3620</a>. It was fixed in celery 4.1. You need multiple tasks in the queue upon initial startup to reproduce this issue. You can also see more discussion about this issue here: <a href="https://github.com/celery/celery/pull/3752" target="_blank">https://github.com/celery/celery/pull/3752</a> and here <a href="https://github.com/celery/celery/issues/4109#issuecomment-314367686" target="_blank">https://github.com/celery/celery/issues/4109#issuecomment-314367686</a>. I implemented this fix in my lab environment and it resolved my issue.</div><div><br></div><div># Problem with pulp_resource_manager service</div><div><div>Jul 25 06:10:03 pulp-lab pulp[17698]: celery.worker:CRITICAL: (17698-58784) File "/usr/lib64/python2.7/site-packages/billiard/pool.py", line 1486, in apply_async</div><div>Jul 25 06:10:03 pulp-lab pulp[17698]: celery.worker:CRITICAL: (17698-58784) self._quick_put((TASK, (result._job, None, func, args, kwds)))</div><div>Jul 25 06:10:03 pulp-lab pulp[17698]: celery.worker:CRITICAL: (17698-58784) TypeError: 'NoneType' object is not callable</div></div><div><br></div><div># Here is the fix that was ultimately implemented which moves the consumption step below the object registration in worker/loops.py.</div><div><div>- consumer.consume()</div><div>- obj.on_ready()</div><div> obj.controller.register_with_event_loop(hub)</div><div> obj.register_with_event_loop(hub)</div><div>+ consumer.consume()</div><div>+ obj.on_ready()</div></div><div><br></div><div><br></div><div>Thanks</div><div><span class="m_-4997864212792679721m_7540986556227809640HOEnZb"><font color="#888888"><div>
<span class="m_-4997864212792679721m_7540986556227809640m_8756559713960247987Apple-style-span" style="border-collapse:separate;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-alternates:normal;font-variant-east-asian:normal;line-height:normal;border-spacing:0px"><div style="word-wrap:break-word"><div>Dustin McNabb</div><div><br></div></div></span><br class="m_-4997864212792679721m_7540986556227809640m_8756559713960247987Apple-interchange-newline"><br class="m_-4997864212792679721m_7540986556227809640m_8756559713960247987Apple-interchange-newline">
</div></font></span><div><div class="m_-4997864212792679721m_7540986556227809640h5">
<div><br><blockquote type="cite"><div>On Jul 24, 2018, at 2:23 PM, Bihan Zhang <<a href="mailto:bizhang@redhat.com" target="_blank">bizhang@redhat.com</a>> wrote:</div><br class="m_-4997864212792679721m_7540986556227809640m_8756559713960247987Apple-interchange-newline"><div><div dir="ltr">Dustin,<div><br></div><div>These fixes were implemented[0] and were just release in pulp 2.16.4 beta and pulp_python 2.0.3 beta today.</div><div>Here's our release schedule in case you want to keep an eye out on when this is going to go GA: <a href="https://pulp.plan.io/projects/pulp/wiki/2164_Release_Status" target="_blank">https://pulp.plan.io/projects/pulp/wiki/2164_Release_Status</a></div><div><br></div><div><br></div><div>[0] <a href="https://pulp.plan.io/issues/3578" target="_blank">https://pulp.plan.io/issues/3578</a></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jul 24, 2018 at 4:50 PM, Dustin McNabb <span dir="ltr"><<a href="mailto:dustin.mcnabb@gmail.com" target="_blank">dustin.mcnabb@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space">I have pulp-server 2.14.3-1 and pulp-python-plugins 2.0.2-1 and I can no longer sync form <a href="http://python.pypi.org/" target="_blank">python.pypi.org</a>. I realize they switched to <a href="http://pypi.org/" target="_blank">pypi.org</a> and <a href="http://files.pythonhosted.org/" target="_blank">files.pythonhosted.org</a>, but the old site still redirects requests. However, the URLs where pulp-python-plugins expects to find packages no longer exist. Also, the pulp-python packages have not been updated since 2017. Does this mean that we don’t have support for the new <a href="http://pypi.org/" target="_blank">pypi.org</a> warehouse using pulp2? Or is there a legacy feed URL I can use?<div><br></div><div><div>Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of <a href="https://pypi.org/simple/pypi/tinydb/json/" target="_blank">https://pypi.org/simple/pypi/tinydb/json/</a> failed with code 404: Not Found</div><div>Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (37): <a href="http://pypi.org/" target="_blank">pypi.org</a></div><div>Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of <a href="https://pypi.org/simple/pypi/setuptools/json/" target="_blank">https://pypi.org/simple/pypi/setuptools/json/</a> failed with code 404: Not Found</div><div>Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (38): <a href="http://pypi.org/" target="_blank">pypi.org</a></div><div>Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of <a href="https://pypi.org/simple/pypi/urllib3/json/" target="_blank">https://pypi.org/simple/pypi/urllib3/json/</a> failed with code 404: Not Found</div><div>Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (39): <a href="http://pypi.org/" target="_blank">pypi.org</a></div><div>Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of <a href="https://pypi.org/simple/pypi/chardet/json/" target="_blank">https://pypi.org/simple/pypi/chardet/json/</a> failed with code 404: Not Found</div><div>Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (40): <a href="http://pypi.org/" target="_blank">pypi.org</a></div><div>Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of <a href="https://pypi.org/simple/pypi/passlib/json/" target="_blank">https://pypi.org/simple/pypi/passlib/json/</a> failed with code 404: Not Found</div><div>Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (41): <a href="http://pypi.org/" target="_blank">pypi.org</a></div><div>Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of <a href="https://pypi.org/simple/pypi/setuptools/json/" target="_blank">https://pypi.org/simple/pypi/setuptools/json/</a> failed with code 404: Not Found</div><div>Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (42): <a href="http://pypi.org/" target="_blank">pypi.org</a></div><div>Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of <a href="https://pypi.org/simple/pypi/gunicorn/json/" target="_blank">https://pypi.org/simple/pypi/gunicorn/json/</a> failed with code 404: Not Found</div><div>Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of <a href="https://pypi.org/simple/pypi/idna/json/" target="_blank">https://pypi.org/simple/pypi/idna/json/</a> failed with code 404: Not Found</div></div><div><br></div><div>Thanks<span class="m_-4997864212792679721m_7540986556227809640m_8756559713960247987HOEnZb"><font color="#888888"><br><div><br class="m_-4997864212792679721m_7540986556227809640m_8756559713960247987m_-4188598782716984740webkit-block-placeholder"></div><div>
<span class="m_-4997864212792679721m_7540986556227809640m_8756559713960247987m_-4188598782716984740Apple-style-span" style="border-collapse:separate;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-alternates:normal;font-variant-east-asian:normal;line-height:normal;border-spacing:0px"><div style="word-wrap:break-word"><div>Dustin McNabb</div><div><br></div></div></span><br class="m_-4997864212792679721m_7540986556227809640m_8756559713960247987m_-4188598782716984740Apple-interchange-newline"><br class="m_-4997864212792679721m_7540986556227809640m_8756559713960247987m_-4188598782716984740Apple-interchange-newline">
</div>
<br></font></span></div></div><br>_______________________________________________<br>
Pulp-list mailing list<br>
<a href="mailto:Pulp-list@redhat.com" target="_blank">Pulp-list@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a><br></blockquote></div><br></div>
</div></blockquote></div><br></div></div></div></div></blockquote></div><br></div>
</div></blockquote></div><br></div></div></div></div><br>_______________________________________________<br>
Pulp-list mailing list<br>
<a href="mailto:Pulp-list@redhat.com" target="_blank">Pulp-list@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a><br></blockquote></div><br></div>
</div></blockquote></div><br></div></div>_______________________________________________<br>
Pulp-list mailing list<br>
<a href="mailto:Pulp-list@redhat.com" target="_blank">Pulp-list@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-list" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-list</a></blockquote></div>