[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Pulp-list] pypi sync no longer working after pypi.org migration with pulp 2.14 and pulp-python-plugins 2.0.2-1



Dustin,
Would you mind opening a pulp issue with a description of the issue and the patch? https://pulp.plan.io/issues/new 
I'm not too familiar with the tasking system, and it would be good for the issue to go through the regular triage process.


Cheers,
Bihan

On Wed, Jul 25, 2018 at 8:18 AM, Dustin McNabb <dustin mcnabb gmail com> wrote:
Bihan -

Thanks for the fast response. However, I’ve upgraded my lab from pulp 2.14 to this beta and it introduces a new problem by upgrading to celery 4.0.2-4. Specifically, we now have this issue https://github.com/celery/celery/issues/3620. It was fixed in celery 4.1. You need multiple tasks in the queue upon initial startup to reproduce this issue. You can also see more discussion about this issue here: https://github.com/celery/celery/pull/3752 and here https://github.com/celery/celery/issues/4109#issuecomment-314367686. I implemented this fix in my lab environment and it resolved my issue.

# Problem with pulp_resource_manager service
Jul 25 06:10:03 pulp-lab pulp[17698]: celery.worker:CRITICAL: (17698-58784)   File "/usr/lib64/python2.7/site-packages/billiard/pool.py", line 1486, in apply_async
Jul 25 06:10:03 pulp-lab  pulp[17698]: celery.worker:CRITICAL: (17698-58784)     self._quick_put((TASK, (result._job, None, func, args, kwds)))
Jul 25 06:10:03 pulp-lab pulp[17698]: celery.worker:CRITICAL: (17698-58784) TypeError: 'NoneType' object is not callable

# Here is the fix that was ultimately implemented which moves the consumption step below the object registration in worker/loops.py.
-    consumer.consume()
-    obj.on_ready()
     obj.controller.register_with_event_loop(hub)
     obj.register_with_event_loop(hub)
+    consumer.consume()
+    obj.on_ready()


Thanks
Dustin McNabb




On Jul 24, 2018, at 2:23 PM, Bihan Zhang <bizhang redhat com> wrote:

Dustin,

These fixes were implemented[0] and were just release in pulp 2.16.4 beta and pulp_python 2.0.3 beta today.
Here's our release schedule in case you want to keep an eye out on when this is going to go GA: https://pulp.plan.io/projects/pulp/wiki/2164_Release_Status



On Tue, Jul 24, 2018 at 4:50 PM, Dustin McNabb <dustin mcnabb gmail com> wrote:
I have pulp-server 2.14.3-1 and pulp-python-plugins 2.0.2-1 and I can no longer sync form python.pypi.org. I realize they switched to pypi.org and files.pythonhosted.org, but the old site still redirects requests. However, the URLs where pulp-python-plugins expects to find packages no longer exist. Also, the pulp-python packages have not been updated since 2017. Does this mean that we don’t have support for the new pypi.org warehouse using pulp2? Or is there a legacy feed URL I can use?

Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://pypi.org/simple/pypi/tinydb/json/ failed with code 404: Not Found
Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (37): pypi.org
Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://pypi.org/simple/pypi/setuptools/json/ failed with code 404: Not Found
Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (38): pypi.org
Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://pypi.org/simple/pypi/urllib3/json/ failed with code 404: Not Found
Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (39): pypi.org
Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://pypi.org/simple/pypi/chardet/json/ failed with code 404: Not Found
Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (40): pypi.org
Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://pypi.org/simple/pypi/passlib/json/ failed with code 404: Not Found
Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (41): pypi.org
Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://pypi.org/simple/pypi/setuptools/json/ failed with code 404: Not Found
Jul 24 20:33:55 pulp-test-server pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (42): pypi.org
Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://pypi.org/simple/pypi/gunicorn/json/ failed with code 404: Not Found
Jul 24 20:33:55 pulp-test-server pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://pypi.org/simple/pypi/idna/json/ failed with code 404: Not Found

Thanks

Dustin McNabb





_______________________________________________
Pulp-list mailing list
Pulp-list redhat com
https://www.redhat.com/mailman/listinfo/pulp-list




[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]