[Pulp-dev] Composed Repositories

[Dennis Kliban]

On Mon, May 14, 2018 at 3:44 PM, Jeff Ortel <jortel at redhat.com> wrote:

> Let's brainstorm on something.
>
> Pulp needs to deal with remote repositories that are composed of multiple
> content types which may span the domain of a single plugin.  Here are a few
> examples.  Some Red Hat RPM repositories are composed of: RPMs, DRPMs, ,
> ISOs and Kickstart Trees.  Some OSTree repositories are composed of OSTrees
> & Kickstart Trees. This raises a question:
>
> How can pulp3 best support syncing with remote repositories that are
> composed of multiple (unrelated) content types in a way that doesn't result
> in plugins duplicating support for content types?
>
> Few approaches come to mind:
>
> 1. Multiple plugins (Remotes) participate in the sync flow to produce a
> new repository version.
> 2. Multiple plugins (Remotes) are sync'd successively each producing a new
> version of a repository.  Only the last version contains the fully sync'd
> composition.
> 3. Plugins share code.
> 4. Other?
>
>
> Option #1: Sync would be orchestrated by core or the user so that multiple
> plugins (Remotes) participate in populating a new repository version.  For
> example: the RPM plugin (Remote) and the Kickstart Tree plugin (Remote)
> would both be sync'd against the same remote repository that is composed of
> both types.  The new repository version would be composed of the result of
> both plugin (Remote) syncs.  To support this, we'd need to provide a way
> for each plugin to operate seamlessly on the same (new) repository
> version.  Perhaps something internal to the RepositoryVersion.  The
> repository version would not be marked "complete" until the last plugin
> (Remote) sync has succeeded.  More complicated than #2 but results in only
> creating truly complete versions or nothing.  No idea how this would work
> with current REST API whereby plugins provide sync endpoints.
>
>
I like this approach because it allows the user to perform a single call to
the REST API and specify multiple "sync methods" to use to create a single
new repository version.


> Option #2: Sync would be orchestrated by core or the user so that multiple
> plugins (Remotes) create successive repository versions.  For example: the
> RPM plugin (Remote) and the Kickstart Tree plugin (Remote) would both be
> sync'd against the same remote repository that is a composition including
> both types.  The intermediate versions would be incomplete.  Only the
> last version contains the fully sync'd composition.  This approach can be
> supported by core today :) but will produce incomplete repository versions
> that are marked complete=True.  This /seems/ undesirable, right?  This may
> not be a problem for distribution since I would imaging that only the last
> (fully composed) version would be published.  But what about other usages
> of the repository's "latest" version?
>
> Option #3: requires a plugin to be aware of specific repository
> composition(s); other plugins and creates a code dependency between
> plugins.  For example, the RPM plugin could delegate ISOs to the File
> plugin and Kickstart Trees to the KickStart Tree plugin.
>
> For all options, plugins (Remotes) need to limit sync to affect only those
> content types within their domain.  For example, the RPM (Remote) sync
> cannot add/remove ISO or KS Trees.
>
> I am an advocate of some from of options #1 or #2.  Combining plugins
> (Remotes) as needed to deal with arbitrary combinations within remote
> repositories seems very powerful; does not impose complexity on plugin
> writers; and does not introduce code dependencies between plugins.
>
> Thoughts?
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180515/83f291ef/attachment.htm>