[Pulp-list] pulp 3.3 migration error

Fri May 29 21:18:00 UTC 2020

Hello Bin,

Unfortunately it looks like when we fixed the migration from 3.3.0 ->
3.3.1, we accidentally broke the migration from 3.2.z -> 3.3.z.  This is my
fault since I tested the various migration paths for that patch - including
migrating from 3.2.0 - but I must have missed something when doing so.

On Monday we will be releasing Pulp RPM 3.4.0 which should fix the
migration issues permanently. We're also considering entirely removing the
3.3.z releases of the RPM plugin from PyPI to avoid anyone else
experiencing this issue going forwards, and investigating automated upgrade
testing to catch this kind of problem earlier.

I apologize for the headache this has probably been, but we really
appreciate the effort you've put into reporting these issues, it is of
tremendous value to the project!

Thank you,
Daniel

On Thu, May 28, 2020 at 11:26 PM Bin Li (BLOOMBERG/ 120 PARK) <
bli111 at bloomberg.net> wrote:

> Yes, we are trying to upgrade to 3.3.2.
>
> From: dkliban at redhat.com At: 05/28/20 17:13:05
> To: Bin Li (BLOOMBERG/ 120 PARK ) <bli111 at bloomberg.net>
> Cc: pulp-list at redhat.com
> Subject: Re: [Pulp-list] pulp 3.3 migration error
>
> Are you upgrading to pulp-rpm 3.3.2?
>
> On Thu, May 28, 2020 at 5:08 PM Bin Li (BLOOMBERG/ 120 PARK) <
> bli111 at bloomberg.net> wrote:
>
>> Hi All,
>> Getting a new error when migrate from 3.2 to 3.3. Please advise if this
>> can be fixed manually.
>>
>> TASK [pulp_database : Run database migrations]
>> ************************************************************************************************************************
>> fatal: [pulpp-ob-581]: FAILED! => {"changed": true, "cmd":
>> ["/opt/utils/venv/pulp/3.7.3/bin/django-admin", "migrate", "--no-input"],
>> "delta": "0:00:01.768909", "end": "2020-05-28 17:02:22.996242", "msg":
>> "non-zero return code", "rc": 1, "start": "2020-05-28 17:02:21.227333",
>> "stderr": "Traceback (most recent call last):\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/utils.py\",
>> line 84, in _execute\n return self.cursor.execute(sql,
>> params)\npsycopg2.errors.NotNullViolation: column
>> \"last_sync_revision_number\" contains null values\n\n\nThe above exception
>> was the direct cause of the following exception:\n\nTraceback (most recent
>> call last):\n File \"/opt/utils/venv/pulp/3.7.3/bin/django-admin\", line 8,
>> in <module>\n sys.exit(execute_from_command_line())\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/management/__init__.py\",
>> line 381, in execute_from_command_line\n utility.execute()\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/management/__init__.py\",
>> line 375, in execute\n
>> self.fetch_command(subcommand).run_from_argv(self.argv)\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/management/base.py\",
>> line 323, in run_from_argv\n self.execute(*args, **cmd_options)\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/management/base.py\",
>> line 364, in execute\n output = self.handle(*args, **options)\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/management/base.py\",
>> line 83, in wrapped\n res = handle_func(*args, **kwargs)\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/management/commands/migrate.py\",
>> line 234, in handle\n fake_initial=fake_initial,\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/migrations/executor.py\",
>> line 117, in migrate\n state = self._migrate_all_forwards(state, plan,
>> full_plan, fake=fake, fake_initial=fake_initial)\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/migrations/executor.py\",
>> line 147, in _migrate_all_forwards\n state = self.apply_migration(state,
>> migration, fake=fake, fake_initial=fake_initial)\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/migrations/executor.py\",
>> line 245, in apply_migration\n state = migration.apply(state,
>> schema_editor)\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/migrations/migration.py\",
>> line 124, in apply\n operation.database_forwards(self.app_label,
>> schema_editor, old_state, project_state)\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/migrations/operations/fields.py\",
>> line 112, in database_forwards\n field,\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/base/schema.py\",
>> line 447, in add_field\n self.execute(sql, params)\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/base/schema.py\",
>> line 137, in execute\n cursor.execute(sql, params)\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/utils.py\",
>> line 67, in execute\n return self._execute_with_wrappers(sql, params,
>> many=False, executor=self._execute)\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/utils.py\",
>> line 76, in _execute_with_wrappers\n return executor(sql, params, many,
>> context)\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/utils.py\",
>> line 84, in _execute\n return self.cursor.execute(sql, params)\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/utils.py\",
>> line 89, in __exit__\n raise dj_exc_value.with_traceback(traceback) from
>> exc_value\n File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/utils.py\",
>> line 84, in _execute\n return self.cursor.execute(sql,
>> params)\ndjango.db.utils.IntegrityError: column
>> \"last_sync_revision_number\" contains null values", "stderr_lines":
>> ["Traceback (most recent call last):", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/utils.py\",
>> line 84, in _execute", " return self.cursor.execute(sql, params)",
>> "psycopg2.errors.NotNullViolation: column \"last_sync_revision_number\"
>> contains null values", "", "", "The above exception was the direct cause of
>> the following exception:", "", "Traceback (most recent call last):", " File
>> \"/opt/utils/venv/pulp/3.7.3/bin/django-admin\", line 8, in <module>", "
>> sys.exit(execute_from_command_line())", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/management/__init__.py\",
>> line 381, in execute_from_command_line", " utility.execute()", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/management/__init__.py\",
>> line 375, in execute", "
>> self.fetch_command(subcommand).run_from_argv(self.argv)", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/management/base.py\",
>> line 323, in run_from_argv", " self.execute(*args, **cmd_options)", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/management/base.py\",
>> line 364, in execute", " output = self.handle(*args, **options)", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/management/base.py\",
>> line 83, in wrapped", " res = handle_func(*args, **kwargs)", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/core/management/commands/migrate.py\",
>> line 234, in handle", " fake_initial=fake_initial,", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/migrations/executor.py\",
>> line 117, in migrate", " state = self._migrate_all_forwards(state, plan,
>> full_plan, fake=fake, fake_initial=fake_initial)", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/migrations/executor.py\",
>> line 147, in _migrate_all_forwards", " state = self.apply_migration(state,
>> migration, fake=fake, fake_initial=fake_initial)", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/migrations/executor.py\",
>> line 245, in apply_migration", " state = migration.apply(state,
>> schema_editor)", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/migrations/migration.py\",
>> line 124, in apply", " operation.database_forwards(self.app_label,
>> schema_editor, old_state, project_state)", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/migrations/operations/fields.py\",
>> line 112, in database_forwards", " field,", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/base/schema.py\",
>> line 447, in add_field", " self.execute(sql, params)", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/base/schema.py\",
>> line 137, in execute", " cursor.execute(sql, params)", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/utils.py\",
>> line 67, in execute", " return self._execute_with_wrappers(sql, params,
>> many=False, executor=self._execute)", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/utils.py\",
>> line 76, in _execute_with_wrappers", " return executor(sql, params, many,
>> context)", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/utils.py\",
>> line 84, in _execute", " return self.cursor.execute(sql, params)", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/utils.py\",
>> line 89, in __exit__", " raise dj_exc_value.with_traceback(traceback) from
>> exc_value", " File
>> \"/opt/utils/venv/pulp/3.7.3/lib64/python3.7/site-packages/django/db/backends/utils.py\",
>> line 84, in _execute", " return self.cursor.execute(sql, params)",
>> "django.db.utils.IntegrityError: column \"last_sync_revision_number\"
>> contains null values"], "stdout": "Operations to perform:\n Apply all
>> migrations: admin, auth, contenttypes, core, file, rpm, sessions\nRunning
>> migrations:\n Applying rpm.0005_optimize_sync...", "stdout_lines":
>> ["Operations to perform:", " Apply all migrations: admin, auth,
>> contenttypes, core, file, rpm, sessions", "Running migrations:", " Applying
>> rpm.0005_optimize_sync..."]}
>> _______________________________________________
>> Pulp-list mailing list
>> Pulp-list at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-list
>
>
> _______________________________________________
> Pulp-list mailing list
> Pulp-list at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-list/attachments/20200529/5ee46df4/attachment.htm>