[Pulp-dev] Import/export: import ordering ideas

[Grant Gainey]

Hey folks,

In the context of 'Pulp Import/Export' development, we've been talking
about how best to solve the problem of import-order. Basically, the
import-process has to have some way of knowing that, to use an example from
pulp_rpm, you don't want to import an UpdateRecord until after you have
imported the Packages that might be pointed to by that update.

We looked at having a given ModelResource override the before_import()
hook, and make sure to invoke the imports for whatever modelresources it
might depend on. The problems were...large, however. The plugin would need
to be able to call back 'out' to the import-process, there would need to be
state managed and perhaps persisted about what models had and had not yet
been imported, and the opportunity for circular-dependencies generated even
more edge-cases to be tested for. It put a large burden on the plugin
author.

What we've come up with is much simpler. To play the import/export game at
all, a plugin must include a <plugin>/app/modelresource.py file that
defines the ModelResource classes for any content that plugin thinks should
be exported. In addition, we've added an ordered list 'IMPORT_ORDER', that
lists that plugin's ModelResources in the order it would like them to be
imported.

You can see an example (of the file and IMPORT_ORDER) here :
https://github.com/pulp/pulp_rpm/pull/1671/files#diff-0a01b735ea130b73ab9ccc39185e7654R124

Would appreciate some more eyes/thought on this before we get too far down
this path. Let us know what you think!

G
-- 
Grant Gainey
Principal Software Engineer, Red Hat System Management Engineering
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20200409/e44728ac/attachment.htm>