[Pulp-dev] Importer and Distributor Modeling

Michael Hrivnak mhrivnak at redhat.com
Thu Sep 8 11:39:19 UTC 2016

I've also been eyeing the opportunity to combine the two catalogs. It makes
a lot of sense.

And if we could reduce an importer's sync responsibility to just creating
units and populating one catalog, and then let pulp do the downloading (at
the appropriate time based on settings), that would give us a common
pattern that most plugins could use.

I'll interested for Jeff to chime in with the reasoning behind the current
Importer and Distributor model.


On Wed, Sep 7, 2016 at 9:01 AM, Jeremy Cline <jcline at redhat.com> wrote:

> On 09/06/2016 04:35 PM, Brian Bouterse wrote:
> > The current Django modeled Importer [0] and Distributor [1] allow for
> > these objects to be associated or unassociated with a repository. I'm
> > familiar with the use cases that motivate associating a
> > importer/distributor with a repository, but what are the use cases
> > around an Importer/Distributor to exist without being associated with a
> > repo? Is there something that we already do which would require this?
> I have been bouncing a few ideas around how to unify a few things. We
> currently have two sets of content catalogs: one for lazy sync, and one
> for alternate content sources. We should really just maintain a single
> catalog for *all* content we handle and that table should be designed
> to support lazy sync, alternate content sources, content scrubbing, etc.
> and it should *always* be built for any content we handle.
> Alternate content sources lets you define a list of places content can
> be obtained, and weights can be applied in order to prefer the least
> expensive option. It requires that "catalogers" be made to inspect the
> content source and decide what content is available for download. This
> is very similar to an importer, which inspects a content source, decides
> what to download, and then downloads it.
> Wouldn't it be nice if you didn't have to write an importer *and* a
> cataloger? Wouldn't it be better to just use an importer to generate
> the catalog and skip downloading the content itself (and does that
> feature sound familiar)? However these alternate content sources aren't
> associated with repositories, so the importer can't be required to
> relate to a repository.
> As far as I know, there is no use-case for a distributor that isn't
> attached to a repository. I believe the justification is that there's
> a group distributor as well as a plain distributor, so there should be
> an intermediate class. However, it is my opinion that we ax the group
> distributor since it's used in a single place, the RPM export code, and
> all it does is loop through a set of repositories and calls their
> repository distributor (potentially with some override configuration).
> This could be done so many other ways with less effort on our part that
> I won't even bother to list them.
> To summarize:
> Let's just have an "Importer" that has an optional relation to a
> repository.
> Let's get rid of the GroupDistributor and turn the
> "RepositoryDistributor" into just "Distributor".
> --
> Jeremy Cline
> XMPP: jeremy at jcline.org
> IRC:  jcline
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20160908/e06418c1/attachment.htm>

More information about the Pulp-dev mailing list