[Pulp-list] Pulp 2.7.0 beta is available

Webb, Reece Reece.Webb at ucsf.edu
Wed Sep 16 18:37:24 UTC 2015 

Hi Randy,

Thanks, I upgraded from 2.6.3. As I said in my last follow-up email, recreating a repository allow it to successfully sync/publish. But now I'm running into two other issues... email notification events don't work at all. I've also run into an error retrieving metadata on seemingly random repositories that I've tried recreating to workaround the offset error. I can recreate the repository, but I receive the same error. I found one bug report with this error that turned out to be a selinux issue, but I have selinux disabled... not good. Logs below.

Thanks,
Reece


--- email notification event errors

Sep 16 18:24:52 yummy pulp: pulp.server.managers.repo.sync:WARNING: Plugin type [yum_importer] on repo [rad-7Server-optional] did not return a valid sync report
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256) Exception from notifier of type [email]
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256) Traceback (most recent call last):
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/event/fire.py", line 76, in _do_fire
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)     f(l['notifier_config'], event)
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)   File "/usr/lib/python2.7/site-packages/pulp/server/event/mail.py", line 35, in handle_event
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)     body = json.dumps(event.data(), indent=2, default=json_util.default)
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)   File "/usr/lib64/python2.7/json/__init__.py", line 250, in dumps
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)     sort_keys=sort_keys, **kw).encode(obj)
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)   File "/usr/lib64/python2.7/json/encoder.py", line 209, in encode
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)     chunks = list(chunks)
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)   File "/usr/lib64/python2.7/json/encoder.py", line 434, in _iterencode
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)     for chunk in _iterencode_dict(o, _current_indent_level):
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)   File "/usr/lib64/python2.7/json/encoder.py", line 408, in _iterencode_dict
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)     for chunk in chunks:
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)   File "/usr/lib64/python2.7/json/encoder.py", line 442, in _iterencode
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)     o = _default(o)
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)   File "/usr/lib64/python2.7/site-packages/bson/json_util.py", line 256, in default
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)     raise TypeError("%%r is not JSON serializable" %% obj)
Sep 16 18:24:52 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256) TypeError: <TaskStatus: TaskStatus object> is not JSON serializable
Sep 16 18:24:52 yummy pulp: pulp.server.async.tasks:INFO: resetting consecutive failure count for schedule None
Sep 16 18:24:52 yummy pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[b6eee9ee-8349-4f93-8581-7fafd11b4b94]
Sep 16 18:24:52 yummy pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.sync.sync[8fea6417-743a-4674-bb94-558f21ab2cd6] succeeded in 4.286946715s: <pulp.server.async.tasks.TaskResult object at 0x75d7a10>
Sep 16 18:24:52 yummy pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[b6eee9ee-8349-4f93-8581-7fafd11b4b94] succeeded in 0.210158428s: None
Sep 16 18:24:52 yummy pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[b26431a3-218a-4cbf-969e-2267a8013ab0] succeeded in 0.248760524s: None
Sep 16 18:24:52 yummy pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.publish.publish[fe941e70-b0c7-4971-84fd-1d2215bb37ea]
Sep 16 18:24:52 yummy pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[c634b5dd-4df3-4198-86e7-ddff4567c4c0]
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256) Exception from notifier of type [email]
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256) Traceback (most recent call last):
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/event/fire.py", line 76, in _do_fire
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)     f(l['notifier_config'], event)
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)   File "/usr/lib/python2.7/site-packages/pulp/server/event/mail.py", line 35, in handle_event
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)     body = json.dumps(event.data(), indent=2, default=json_util.default)
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)   File "/usr/lib64/python2.7/json/__init__.py", line 250, in dumps
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)     sort_keys=sort_keys, **kw).encode(obj)
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)   File "/usr/lib64/python2.7/json/encoder.py", line 209, in encode
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)     chunks = list(chunks)
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)   File "/usr/lib64/python2.7/json/encoder.py", line 434, in _iterencode
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)     for chunk in _iterencode_dict(o, _current_indent_level):
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)   File "/usr/lib64/python2.7/json/encoder.py", line 408, in _iterencode_dict
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)     for chunk in chunks:
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)   File "/usr/lib64/python2.7/json/encoder.py", line 442, in _iterencode
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)     o = _default(o)
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)   File "/usr/lib64/python2.7/site-packages/bson/json_util.py", line 256, in default
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256)     raise TypeError("%%r is not JSON serializable" %% obj)
Sep 16 18:24:54 yummy pulp: pulp.server.managers.event.fire:ERROR: (2016-40256) TypeError: <TaskStatus: TaskStatus object> is not JSON serializable
Sep 16 18:24:54 yummy pulp: pulp.server.async.tasks:INFO: resetting consecutive failure count for schedule None
Sep 16 18:24:54 yummy pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.publish.publish[fe941e70-b0c7-4971-84fd-1d2215bb37ea] succeeded in 1.803173554s: {'exception': None, 'repo_id': u'rad-7Server-optional', 'traceback': None, 'started': '2015-09-16T18:24:52Z', '_ns':...
Sep 16 18:24:54 yummy pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[c634b5dd-4df3-4198-86e7-ddff4567c4c0] succeeded in 0.0632455370001s: None




--- metadata error - recreating repository doesn't fix this one. sync worked fine prior to upgrade to 2.7.

# pulp-admin rpm repo sync run --repo-id rad-7Server
+----------------------------------------------------------------------+
                 Synchronizing Repository [rad-7Server]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.

Task Failed

Error retrieving metadata: Not found



- messages
Sep 16 18:17:46 yummy pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[347ab5c3-1f47-4edb-88a0-d3be5e248c62]
Sep 16 18:17:46 yummy pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.sync.sync[af75f802-8ea8-4285-8c47-ce5671ea0316]
Sep 16 18:17:46 yummy pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[f0585d85-5ffb-4617-ae60-34b4091e2e33]
Sep 16 18:17:46 yummy pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[347ab5c3-1f47-4edb-88a0-d3be5e248c62] succeeded in 0.256292566s: None
Sep 16 18:17:46 yummy pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os/.
Sep 16 18:17:46 yummy pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Sep 16 18:17:47 yummy pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Sep 16 18:17:47 yummy pulp: pulp.server.async.tasks:INFO: Task failed : [af75f802-8ea8-4285-8c47-ce5671ea0316] : Error retrieving metadata: Not found
Sep 16 18:17:48 yummy pulp: celery.worker.job:INFO: Task pulp.server.managers.repo.sync.sync[af75f802-8ea8-4285-8c47-ce5671ea0316] raised expected: PulpCodedException()
Sep 16 18:17:48 yummy pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[f0585d85-5ffb-4617-ae60-34b4091e2e33] succeeded in 0.0707463489998s: None








On 9/16/15, 10:29 AM, "pulp-list-bounces at redhat.com on behalf of Randy Barlow" <pulp-list-bounces at redhat.com on behalf of rbarlow at redhat.com> wrote:

>Webb, Reece wrote:
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784) Task pulp.server.managers.repo.publish.publish[e325dd8a-3541-4f10-a5f0-80f6a51f8a19] raised unexpected: TypeError("can't compare offset-naive and offset-aware datetimes",)
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784) Traceback (most recent call last):
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784)     R = retval = fun(*args, **kwargs)
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 395, in __call__
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784)     return super(Task, self).__call__(*args, **kwargs)
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784)     return self.run(*args, **kwargs)
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py", line 88, in publish
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784)     transfer_repo, conduit, call_config)
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py", line 116, in _do_publish
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784)     publish_report = publish_repo(transfer_repo, conduit, call_config)
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 567, in wrap_f
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784)     return f(*args, **kwargs)
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 170, in publish_repo
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784)     self._publisher = publish.Publisher(repo, publish_conduit, config, TYPE_ID_DISTRIBUTOR_YUM)
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 257, in __init__
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784)     ((last_deleted and last_published > last_deleted) or not last_deleted):
>> Sep 16 08:26:19 yummy pulp: celery.worker.job:ERROR: (1822-58784) TypeError: can't compare offset-naive and offset-aware datetimes
>
>Hello Reece! This looks like a bug, so I went ahead and filed it for you
>here:
>
>https://pulp.plan.io/issues/1254
>
>What version of Pulp did you upgrade from? Feel free to add that info to
>the bug, or you can reply here if that's more convenient. Thanks for
>letting us know about the issue!
>

More information about the Pulp-list mailing list