[Pulp-list] pulp2 migration: AccessPolicy matching query does not exist

Tatiana Tereshchenko ttereshc at redhat.com
Thu Oct 15 12:02:41 UTC 2020 

Good to know that it's not for the upgrade.

We've planned to have a way to restart migration from scratch but never got
to it and no one asked before.
Here is a story I filed, feel free to provide any feedback there
https://pulp.plan.io/issues/7714.

It would be helpful for us to understand your use case.
Could you share why you wanted to start completely from scratch and not
just re-run the existing migration plan or run a new one?

Thanks,
Tanya

On Wed, Oct 14, 2020 at 7:07 PM Winberg Adam <Adam.Winberg at smhi.se> wrote:

> > probably because we never ask to wipe out the database for upgrades.
>
>
> My reason for doing a 'flush' was to rerun my pulp2 migration from
> scratch, so not really because of the upgrade. But with the background
> provided by you, issue https://pulp.plan.io/issues/6963 now makes more
> sense to me. I guess I should preferably use the 'teardown' util mentioned
> there instead? It is however unclear to me how to use that.
>
>
> The 'reset_db' won't work for me since we have a centralized postgres
> infrastructure where I don't have permissions to drop/create db's (I had to
> get the help of a DBMS to drop my pulp-db).
>
>
> @Brian Bouterse  - thanks for creating the issue!
>
>
> //Adam
>
>
>
>
>
> ------------------------------
> *From:* Brian Bouterse <bmbouter at redhat.com>
> *Sent:* 14 October 2020 18:17
> *To:* Tatiana Tereshchenko
> *Cc:* Winberg Adam; pulp-list at redhat.com
> *Subject:* Re: [Pulp-list] pulp2 migration: AccessPolicy matching query
> does not exist
>
> I've filed this issue tracking the improvement which would allow users to
> run `flush` and not experience this problem.
>
> https://pulp.plan.io/issues/7710
>
> On Wed, Oct 14, 2020 at 12:14 PM Tatiana Tereshchenko <ttereshc at redhat.com>
> wrote:
>
>> Adam,
>> I agree we lack documentation on resetting the environment and the
>> database, probably because we never ask to wipe out the database for
>> upgrades.
>> The instructions are usually provided with release and ask you basically
>> to run the latest pulp_installer.
>>
>> There is some data which is provided with migrations and which has to be
>> present in the database, that's why dropping the database and applying
>> migrations work and `flush` does not.
>> `flush` just removes data and keeps all the tables in place, so
>> migrations are not re-applied.
>> So for now, please do not use `flush` if you want to start using pulp 3
>> db from scratch. We potentially can provide a separate command or find some
>> other way to fill in the essential data back.
>> Please file an issue in our tracker if such feature/command is helpful
>> for you. https://pulp.plan.io/projects/pulp/issues/new
>>
>> As a side note, if you are interested, reset_db command drops database.
>> It's provided as a part of django-extensions package
>> https://django-extensions.readthedocs.io/en/latest/command_extensions.html
>>
>> I'm glad that the migration  runs for you now,
>> Tanya
>>
>>
>> On Wed, Oct 14, 2020 at 3:52 PM Winberg Adam <Adam.Winberg at smhi.se>
>> wrote:
>>
>>> Thanks for the reply!
>>>
>>>
>>> I use 'flush' to clear the db, I don't have a 'reset_db' command.
>>> Otherwise that's pretty much my process.
>>>
>>>
>>> The migrate command returns 'No migrations to apply'.
>>>
>>> Running 'pulpcore-manager show-migrations' shows that all migrations,
>>> including 'guardian.0001/guardian.0002' has checkmarks ([X]).
>>>
>>> guardian
>>>  [X] 0001_initial
>>>  [X] 0002_generic_permissions_index
>>>
>>> The creation of the migration plan works so I assume my admin user is
>>> ok.
>>>
>>>
>>> I am using an rpm based installation on RHEL8, with
>>>
>>> python3-pulp-2to3-migration-0.4.0-1.el8.noarch
>>> python3-pulp-rpm-3.7.0-1.el8.noarch
>>> python3-pulpcore-3.7.1-3.el8.noarch
>>>
>>>
>>> I don't know what went wrong, but I surrendered and dropped my DB and
>>> redid the migrations from scratch - and now it works.
>>> Is there a documented instruction on upgrading existing installations?
>>>
>>> //Adam
>>>
>>>
>>>
>>> ------------------------------
>>> *From:* Tatiana Tereshchenko <ttereshc at redhat.com>
>>> *Sent:* 14 October 2020 14:15
>>> *To:* Winberg Adam
>>> *Cc:* pulp-list at redhat.com
>>> *Subject:* Re: [Pulp-list] pulp2 migration: AccessPolicy matching query
>>> does not exist
>>>
>>>
>>>
>>> On Wed, Oct 14, 2020 at 2:12 PM Tatiana Tereshchenko <
>>> ttereshc at redhat.com> wrote:
>>>
>>>> Hi Adam,
>>>>
>>>> My understanding is that you did the following:
>>>>  * stop pulp services
>>>>  * pulpcore-manager (or django-admin) reset_db
>>>>  * pulpcore-manager migrate
>>>>  * pulpcore-manager reset-admin-password --password password
>>>>  * start services
>>>>  * http POST :/pulp/api/v3/migration-plans/ < your_migraiton_plan.json
>>>>  * http POST
>>>> :/pulp/api/v3/migration-plans/48d03a72-96a1-4d36-9f8b-9a57e97846ef/run/
>>>>
>>>
>>> Sent too early :)
>>> I can't reproduce it so far, so any hints about what can be special
>>> about your environment or installation would be appreciated.
>>> Make sure that you have at least one user which has admin privileges and
>>> that the guardian migrations ran indeed.
>>>   Applying guardian.0001_initial... OK
>>>   Applying guardian.0002_generic_permissions_index... OK
>>>
>>> Tanya
>>>
>>>
>>>
>>>
>>>
>>>>
>>>> On Wed, Oct 14, 2020 at 8:02 AM Winberg Adam <Adam.Winberg at smhi.se>
>>>> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>>
>>>>> so I updated my pulp3 installation from 3.4 to 3.7 and tried to rerun
>>>>> my pulp2 migration - but it errors out with "AccessPolicy matching
>>>>> query does not exist". Anyone know why?
>>>>>
>>>>>
>>>>> I flushed my db, reran the 'migrate' job, created a pulp2migration
>>>>> plan (which worked fine) and then tried to run it. Here's the complete
>>>>> error:
>>>>>
>>>>>
>>>>> Oct 14 05:43:26  gunicorn[2150852]: pulp: django.request:ERROR:
>>>>> Internal Server Error:
>>>>> /pulp/api/v3/migration-plans/48d03a72-96a1-4d36-9f8b-9a57e97846ef/run/
>>>>> Oct 14 05:43:26  gunicorn[2150852]: Traceback (most recent call last):
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/django/core/handlers/exception.py", line
>>>>> 34, in inner
>>>>> Oct 14 05:43:26  gunicorn[2150852]:     response =
>>>>> get_response(request)
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/django/core/handlers/base.py", line 115,
>>>>> in _get_response
>>>>> Oct 14 05:43:26  gunicorn[2150852]:     response =
>>>>> self.process_exception_by_middleware(e, request)
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/django/core/handlers/base.py", line 113,
>>>>> in _get_response
>>>>> Oct 14 05:43:26  gunicorn[2150852]:     response =
>>>>> wrapped_callback(request, *callback_args, **callback_kwargs)
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/django/views/decorators/csrf.py", line
>>>>> 54, in wrapped_view
>>>>> Oct 14 05:43:26  gunicorn[2150852]:     return view_func(*args,
>>>>> **kwargs)
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/rest_framework/viewsets.py", line 114, in
>>>>> view
>>>>> Oct 14 05:43:26  gunicorn[2150852]:     return self.dispatch(request,
>>>>> *args, **kwargs)
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 505, in
>>>>> dispatch
>>>>> Oct 14 05:43:26  gunicorn[2150852]:     response =
>>>>> self.handle_exception(exc)
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 465, in
>>>>> handle_exception
>>>>> Oct 14 05:43:26  gunicorn[2150852]:
>>>>>  self.raise_uncaught_exception(exc)
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 476, in
>>>>> raise_uncaught_exception
>>>>> Oct 14 05:43:26  gunicorn[2150852]:     raise exc
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 502, in
>>>>> dispatch
>>>>> Oct 14 05:43:26  gunicorn[2150852]:     response = handler(request,
>>>>> *args, **kwargs)
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/viewsets.py",
>>>>> line 85, in run
>>>>> Oct 14 05:43:26  gunicorn[2150852]:     'dry_run': dry_run
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/pulpcore/tasking/tasks.py", line 236, in
>>>>> enqueue_with_reservation
>>>>> Oct 14 05:43:26  gunicorn[2150852]:     **parent_kwarg,
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in
>>>>> manager_method
>>>>> Oct 14 05:43:26  gunicorn[2150852]:     return
>>>>> getattr(self.get_queryset(), name)(*args, **kwargs)
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 422, in
>>>>> create
>>>>> Oct 14 05:43:26  gunicorn[2150852]:     obj.save(force_insert=True,
>>>>> using=self.db)
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/django_lifecycle/mixins.py", line 132, in
>>>>> save
>>>>> Oct 14 05:43:26  gunicorn[2150852]:
>>>>>  self._run_hooked_methods(AFTER_CREATE)
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/django_lifecycle/mixins.py", line 207, in
>>>>> _run_hooked_methods
>>>>> Oct 14 05:43:26  gunicorn[2150852]:     method()
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/django_lifecycle/decorators.py", line 69,
>>>>> in func
>>>>> Oct 14 05:43:26  gunicorn[2150852]:     hooked_method(*args, **kwargs)
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/pulpcore/app/models/access_policy.py",
>>>>> line 60, in add_perms
>>>>> Oct 14 05:43:26  gunicorn[2150852]:     access_policy =
>>>>> AccessPolicy.objects.get(viewset_name=self.ACCESS_POLICY_VIEWSET_NAME)
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in
>>>>> manager_method
>>>>> Oct 14 05:43:26  gunicorn[2150852]:     return
>>>>> getattr(self.get_queryset(), name)(*args, **kwargs)
>>>>> Oct 14 05:43:26  gunicorn[2150852]:   File
>>>>> "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 408, in
>>>>> get
>>>>> Oct 14 05:43:26  gunicorn[2150852]:     self.model._meta.object_name
>>>>> Oct 14 05:43:26  gunicorn[2150852]:
>>>>> pulpcore.app.models.access_policy.AccessPolicy.DoesNotExist: AccessPolicy
>>>>> matching query does not exist.
>>>>>
>>>>>
>>>>> Regards
>>>>>
>>>>> //Adam
>>>>> _______________________________________________
>>>>> Pulp-list mailing list
>>>>> Pulp-list at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/pulp-list
>>>>
>>>> _______________________________________________
>> Pulp-list mailing list
>> Pulp-list at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20201015/0fc8cee3/attachment.htm>

More information about the Pulp-list mailing list