[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Pulp-dev] Replicate metadata



I think it would resolve like this:

If repository.exact_mirror == True:
  # sync as if sync_mode='mirror' and also save the remote metadata as a unit also so the NoMetadataPublisher can generically publish it
else:
  # sync according to sync_mode either 'additive' or 'mirror'. This mirror will *not* save the remote metadata so that's how it's feature different

I can write this up in redmine and send it to the list tomorrow.


On Wed, May 16, 2018 at 2:58 PM, David Davis <daviddavis redhat com> wrote:
I had a similar idea in mind. How would this work with the sync_mode option though?

+1 to adding this to redmine.


David

On Wed, May 16, 2018 at 2:40 PM, Brian Bouterse <bbouters redhat com> wrote:
I was thinking about these use cases. I'll use the same "first" and "second" that @dkliban described.

I recognize the "first" use case as a valid one. I think we could bring it to bear pretty easily with a little bit of plugin involvement. We could have a repository attribute boolen called exact_mirror (or similar) which when False would do the following:
(a) allow a plugin writer to implement the exact-saving of metadata as an Artifact
(b) disable add/remove of content using the always-raise an exception with this proposal here:  https://pulp.plan.io/issues/3541#note-3
(c) have the plugin API offer a generic publisher called NoMetdataPublisher that blindly publishes all Artifacts associated with a RepoVersion

This would cause the repo and its metadata to get published exactly, and prevent core from corrupting it by changing the other content since we can't also change the metadata.

For the "second" use case, I think you could fulfil it by making an exact mirror of the orignal publish so that it's only published once. This adds a nice mirroring capability to Pulp.

Should we move these plans into Redmine? What else do we need to know?


On Mon, May 14, 2018 at 3:12 PM, Dennis Kliban <dkliban redhat com> wrote:
On Mon, May 14, 2018 at 1:27 PM, Justin Sherrill <jsherril redhat com> wrote:

I think it kind depends in how pulp would do it.  Thinking of the yum example, all the information is there in an upstream yum repo for pulp to import the 'publication' as is.  If it can be done 'naturally' with a yum repo, there wouldn't be anything special around pulp -> pulp, it'd just be  yum_repo -> pulp.  However we'd need a pulp dev to chime in here :)

This workflow is two use cases in Pulp 3:

   - "As a user, I can create a repository version that is an exact mirror of a remote."
   - "As a user, I can publish a repository version without generating metadata."

The first use case would be satisfied by having the plugin provide code that downloads the metadata from the remote and saves it as 'metadata content' that's part of the repository version.

The second use case could be satisfied by pulpcore providing a generic publisher that pubilshes everything in a repository version. The metadata would just be treated like any other files in the repo.

 

Justin


On 05/14/2018 12:08 PM, Bryan Kearney wrote:
@Justin, I think that makes sense for Pulp->Pulp. For Matthias, I think
he needs Native->Pulp which would not have a publication.

-- bk

On 05/14/2018 11:42 AM, Matthias Dellweg wrote:
Mirroring the metedata exactly is also very important for Debian
Repositories, because of the way the metadata is signed in lieu of the
whole content. So it would be very beneficial, if this could be
provided as a 'service' of the pulp core platform somehow.

Matthias

On Mon, 14 May 2018 11:31:52 -0400
Justin Sherrill <jsherril redhat com> wrote:

 From my understanding of pulp 3, this would maybe involve the
ability to 'import' a publication.  Would that make sense?

Justin


On 05/09/2018 08:22 AM, Bryan Kearney wrote:
One of the themes I heard yesterday at the Red Hat Summit was around
having a pulp server mirror the upstream RPM repo metadata exactly.
The use case is that two pulp servers are behind a load balancer
mirroring the same repo. The users would like to be able to flip a
yum client acrross the two servers. Running createrepo to make
unique repos causes issues for the clients that appear to be
errors. I assume this pattern would not be unique for other package
clients that cache metadata.

So, when looking ahead to pulp 3 I would ask that this be taken into
consideration. I can provide more info / use cases if necessary.

-- bk



_______________________________________________
Pulp-dev mailing list
Pulp-dev redhat com
https://www.redhat.com/mailman/listinfo/pulp-dev  
_______________________________________________
Pulp-dev mailing list
Pulp-dev redhat com
https://www.redhat.com/mailman/listinfo/pulp-dev

      

_______________________________________________
Pulp-dev mailing list
Pulp-dev redhat com
https://www.redhat.com/mailman/listinfo/pulp-dev


_______________________________________________
Pulp-dev mailing list
Pulp-dev redhat com
https://www.redhat.com/mailman/listinfo/pulp-dev



_______________________________________________
Pulp-dev mailing list
Pulp-dev redhat com
https://www.redhat.com/mailman/listinfo/pulp-dev



_______________________________________________
Pulp-dev mailing list
Pulp-dev redhat com
https://www.redhat.com/mailman/listinfo/pulp-dev




[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]