[Pulp-dev] Replicate metadata

[Brian Bouterse]

I was thinking about these use cases. I'll use the same "first" and
"second" that @dkliban described.

I recognize the "first" use case as a valid one. I think we could bring it
to bear pretty easily with a little bit of plugin involvement. We could
have a repository attribute boolen called exact_mirror (or similar) which
when False would do the following:
(a) allow a plugin writer to implement the exact-saving of metadata as an
Artifact
(b) disable add/remove of content using the always-raise an exception with
this proposal here:  https://pulp.plan.io/issues/3541#note-3
(c) have the plugin API offer a generic publisher called NoMetdataPublisher
that blindly publishes all Artifacts associated with a RepoVersion

This would cause the repo and its metadata to get published exactly, and
prevent core from corrupting it by changing the other content since we
can't also change the metadata.

For the "second" use case, I think you could fulfil it by making an exact
mirror of the orignal publish so that it's only published once. This adds a
nice mirroring capability to Pulp.

Should we move these plans into Redmine? What else do we need to know?


On Mon, May 14, 2018 at 3:12 PM, Dennis Kliban <dkliban at redhat.com> wrote:

> On Mon, May 14, 2018 at 1:27 PM, Justin Sherrill <jsherril at redhat.com>
> wrote:
>
>> I think it kind depends in how pulp would do it.  Thinking of the yum
>> example, all the information is there in an upstream yum repo for pulp to
>> import the 'publication' as is.  If it can be done 'naturally' with a yum
>> repo, there wouldn't be anything special around pulp -> pulp, it'd just be
>> yum_repo -> pulp.  However we'd need a pulp dev to chime in here :)
>>
>> This workflow is two use cases in Pulp 3:
>
>    - "As a user, I can create a repository version that is an exact mirror
> of a remote."
>    - "As a user, I can publish a repository version without generating
> metadata."
>
> The first use case would be satisfied by having the plugin provide code
> that downloads the metadata from the remote and saves it as 'metadata
> content' that's part of the repository version.
>
> The second use case could be satisfied by pulpcore providing a generic
> publisher that pubilshes everything in a repository version. The metadata
> would just be treated like any other files in the repo.
>
>
>
>> Justin
>>
>> On 05/14/2018 12:08 PM, Bryan Kearney wrote:
>>
>> @Justin, I think that makes sense for Pulp->Pulp. For Matthias, I think
>> he needs Native->Pulp which would not have a publication.
>>
>> -- bk
>>
>> On 05/14/2018 11:42 AM, Matthias Dellweg wrote:
>>
>> Mirroring the metedata exactly is also very important for Debian
>> Repositories, because of the way the metadata is signed in lieu of the
>> whole content. So it would be very beneficial, if this could be
>> provided as a 'service' of the pulp core platform somehow.
>>
>> Matthias
>>
>> On Mon, 14 May 2018 11:31:52 -0400
>> Justin Sherrill <jsherril at redhat.com> <jsherril at redhat.com> wrote:
>>
>>
>>  From my understanding of pulp 3, this would maybe involve the
>> ability to 'import' a publication.  Would that make sense?
>>
>> Justin
>>
>>
>> On 05/09/2018 08:22 AM, Bryan Kearney wrote:
>>
>> One of the themes I heard yesterday at the Red Hat Summit was around
>> having a pulp server mirror the upstream RPM repo metadata exactly.
>> The use case is that two pulp servers are behind a load balancer
>> mirroring the same repo. The users would like to be able to flip a
>> yum client acrross the two servers. Running createrepo to make
>> unique repos causes issues for the clients that appear to be
>> errors. I assume this pattern would not be unique for other package
>> clients that cache metadata.
>>
>> So, when looking ahead to pulp 3 I would ask that this be taken into
>> consideration. I can provide more info / use cases if necessary.
>>
>> -- bk
>>
>>
>>
>> _______________________________________________
>> Pulp-dev mailing listPulp-dev at redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev
>>
>> _______________________________________________
>> Pulp-dev mailing listPulp-dev at redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>>
>> _______________________________________________
>> Pulp-dev mailing listPulp-dev at redhat.comhttps://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>>
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180516/e63d42e4/attachment.htm>