<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Distributions definitely do not need to be migrated (i see them
more as configuration similar to importers, than i do as something
that can be exported/imported).</p>
<p>Publications i think would be a nice to have, but can be
re-generated on the other side based on the repository version.
While the publication won't be 100% exact (due to timestamps and
ordering differences), i'm not sure that it matters. The biggest
drawback will just be that the importing side needs to be smarter,
and it will take a little longer. I could see import/export of
publications being an optimization or extension later on, but not
necessary for a first pass. If i'm missing some issue or use
case, please do speak up!</p>
<p>Justin<br>
</p>
<div class="moz-cite-prefix">On 2/17/20 1:05 PM, David Davis wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAHa=2WkNyH+bDr-RGqvxJWWwdFdW0ryZe9f0BGwt+a51SbHcjw@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">The user stories that Katello gave us (which I've
entered into redmine here[0]) don't mention publications or
distributions. I will follow up with Katello though.
<div><br>
</div>
<div>
<div>[0] <a href="https://pulp.plan.io/issues/6134"
target="_blank" moz-do-not-send="true">https://pulp.plan.io/issues/6134</a><br
clear="all">
<div>
<div dir="ltr" data-smartmail="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div><br>
</div>
<div>David<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Feb 17, 2020 at 12:49
PM Dennis Kliban <<a href="mailto:dkliban@redhat.com"
target="_blank" moz-do-not-send="true">dkliban@redhat.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Feb 14, 2020 at
1:11 PM David Davis <<a
href="mailto:daviddavis@redhat.com" target="_blank"
moz-do-not-send="true">daviddavis@redhat.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div>Grant and I met today to discuss importers and
exporters[0] and we'd like some feedback before we
proceed with the design. To sum up this feature
briefly: users can export a repository version from
one Pulp instance and import it to another. </div>
</div>
</blockquote>
<div><br>
</div>
<div>Exporting just repository versions is not sufficient
for reproducing a Pulp instance in an air gapped
environment. Users need to be able to use the "export"
to populate a Pulp instance without needing to create
any publications and/or distributions afterwards. What
about letting users specify a repository to export and
then have pulpcore figure out which repository versions,
publications, distributions, content, metadata, and
artifacts need to be exported? <br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div><br>
</div>
# Master/Detail vs Core
<div><br>
</div>
<div>So one fundamental question is whether we should
use a Master/Detail approach or just have core
control the flow but call out to plugins to get
export formats.</div>
<div><br>
</div>
<div>To give some background: we currently define
Exporters (ie FileSystemExporter) in core as Master
models. Plugins extend this model which allows them
to configure or customize the Exporter. This was
necessary because some plugins need to export
Publications (along with repository metadata) while
other plugins who don't have Publications or
metadata export RepositoryVersions.</div>
<div><br>
</div>
<div>The other option is to have core handle the
workflow. The user would call a core endpoint and
provide a RepositoryVersion. This would work because
for importing/exporting, you wouldn't ever use
Publications because metadata won't be used for
importing back into Pulp. If needed, core could
provide a way for plugin writers to write custom
handlers/exporters for content types.</div>
<div><br>
</div>
<div>If we go with the second option, the question
then becomes whether we should divorce the concept
of Exporters and import/export. Or do we also switch
Exporters from Master/Detail to core only?</div>
<div><br>
</div>
<div># Foreign Keys</div>
<div><br>
</div>
<div>Content can be distributed across multiple tables
(eg UpdateRecord has UpdateCollection, etc). In our
export, we could either use primary keys (UUIDs) or
natural keys to relate records. The former assumes
that UUIDs are unique across Pulp instances. The
safer but more complex alternative is to use natural
keys. This would involve storing a set of fields on
a record that would be used to identify a related
record.</div>
<div><br>
</div>
<div># Incremental Exports</div>
<div><br>
</div>
<div>There are two big pieces of data contained in an
export: the dataset of Content from the database and
the artifact files. An incremental export cuts down
on the size of an export by only exporting the
differences. However, when performing an incremental
export, we could still export the complete dataset
instead of just a set of differences
(additions/removals/updates). This approach would be
simpler and it would allow us to ensure that the new
repo version matches the exported repo version
exactly. It would however increase the export size
but not by much I think--probably some number of
megabytes at most.</div>
<div><br>
</div>
<div>[0] <a href="https://pulp.plan.io/issues/6134"
target="_blank" moz-do-not-send="true">https://pulp.plan.io/issues/6134</a></div>
<div><br>
</div>
<div>
<div>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>David<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
_______________________________________________<br>
Pulp-dev mailing list<br>
<a href="mailto:Pulp-dev@redhat.com" target="_blank"
moz-do-not-send="true">Pulp-dev@redhat.com</a><br>
<a
href="https://www.redhat.com/mailman/listinfo/pulp-dev"
rel="noreferrer" target="_blank"
moz-do-not-send="true">https://www.redhat.com/mailman/listinfo/pulp-dev</a><br>
</blockquote>
</div>
</div>
</blockquote>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Pulp-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>
<a class="moz-txt-link-freetext" href="https://www.redhat.com/mailman/listinfo/pulp-dev">https://www.redhat.com/mailman/listinfo/pulp-dev</a>
</pre>
</blockquote>
</body>
</html>