[Pulp-dev] Export-by-repositoryversion : design thinking needed!

[David Davis]

David


On Mon, Apr 27, 2020 at 3:16 PM Grant Gainey <ggainey at redhat.com> wrote:

> Discussions with Katello uncovered a use-case that we thought about
> briefly in the initial design of import/export, and then put aside to get
> to first-light. That issue is "must be able to export a specific set of
> RepositoryVersions for the specified Repositories" (as opposed to "export
> current-version always"). This is a large use case for katello's
> lifecycle-workflows.
>
> In the design doc this is mentioned under 'Exporter' - the implication
> being that if one wants this, one creates a new Exporter every time. This
> is...not a great approach.
>
> Exporter and Importer work at the Repository level. This is so that the
> user can map upstream and downstream repositories once, at
> Importer-create-time, and then reuse that Importer repeatedly. If an
> Exporter is a single-use entity, as this approach would imply, it requires
> the user to do that startup work every time.
>
> A better (imho) approach is to allow a set of RepositoryVersions to be
> specified at Export-creation time (by, say, providing a
> repositories=[<href-list.] parameter to POST .../exports/) . The impact on
> the Export process is minimal - where now it runs down the list of
> Repositories in the Exporter and gets latest-version from each, it would
> just take the provided list instead.
>

+1


>
> There is an extra validation step needed.  The provided RepositoryVersions *MUST
> *be *1-to-1* associated with the Repositories on the Exporter, or Import
> isn't going to know what to do with them. (must be 1-to-1, but not
> necessarily 'onto' - you may specify fewer RepositoryVersions than you have
> Repositories, it just means the missed-Repositories aren't provided in the
> Export)
>
> There is an open question: How does this interact with incrementals and
> last_export? Current implementation assumes that today's export is going to
> be 'newer than' the last time an Exporter fired. and an incremental is the
> difference, for a given Repository, between the RepoVersion in last_export
> and today's state. Allowing export-by-specific-version might break that,
> will need to do some POC work to convince myself one way or another.
>

Assume the case where you export versions of repos B, C for an exporter
that has A, B, C. I would assume the exported resources for the export will
have only B and C. What happens the next time you do an incremental export
for all repos? What would be the base version (not sure what to call it)
you would get for A? I suppose there are three options:

1. Version 0 of A (ie a full export of A)
2. We look back through exports for the last version of A that was exported
and use it as the base
3. We forbid users from exporting a subset of repos. Instead they should
create a new Exporter if they want to export repos B and C

Any other options?


>
> I have a nagging feeling that we are still missing something about this
> use case. I definitely need more viewpoints on this; can I get
> comments/thoughts by COB 29-APR? This is pretty important to be flattened
> to support katello's workflows.
>
> Thanks for reading this far!
> G
> --
> Grant Gainey
> Principal Software Engineer, Red Hat System Management Engineering
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20200428/671bc854/attachment.htm>