<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<br>
<br>
<div class="moz-cite-prefix">On 05/15/2018 05:58 AM, Austin
Macdonald wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CA+Kyt=q9Xv5TOs_yCGNvbJHb9zavoSrkY72uqk6-wJ4wJLoX8A@mail.gmail.com">
<div dir="auto">
<div>Here's another complexity, how do 2 plugins create a single
publication? </div>
</div>
</blockquote>
<br>
The plugin API could make this seamless. <br>
<br>
<blockquote type="cite"
cite="mid:CA+Kyt=q9Xv5TOs_yCGNvbJHb9zavoSrkY72uqk6-wJ4wJLoX8A@mail.gmail.com">
<div dir="auto">
<div>We basically have the same problem of 2 parallel operations
creating content from a single source. <br>
</div>
</div>
</blockquote>
<br>
I don't think so. plugins should not manipulate content outside of
their domain (other plugins content) so either serial or parallel
should be safe.<br>
<br>
<blockquote type="cite"
cite="mid:CA+Kyt=q9Xv5TOs_yCGNvbJHb9zavoSrkY72uqk6-wJ4wJLoX8A@mail.gmail.com">
<div dir="auto">
<div><br>
<div class="gmail_quote">
<div dir="ltr">On Tue, May 15, 2018, 06:27 Ina Panova <<a
href="mailto:ipanova@redhat.com" target="_blank"
rel="noreferrer" moz-do-not-send="true">ipanova@redhat.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>+1 on not introducing dependencies between
plugins.<br>
<br>
</div>
What will be the behavior in case there is a
composed repo of rpm and ks trees but just the
rpm plugin is installed?</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
<div dir="auto">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>
<div>
<div>Do we fail and say we cannot sync this repo at
all or we just sync the rpm part?<br>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
<div dir="auto"><br>
</div>
<div dir="auto">
<div dir="auto" style="font-family:sans-serif">Assuming
plugins do not depend on each other, I think that when each
plugin looks at the upstream repo, they will only "see" the
content of that type. Conceptually, we will have 2 remotes,
so it will feel like we are syncing from 2 totally distinct
repositories.</div>
<div dir="auto" style="font-family:sans-serif"><br>
</div>
<div dir="auto" style="font-family:sans-serif">The solution
I've been imagining is a lot like 2. Each plugin would sync
to a *separate repository.* These separate repositories are
then published creating *separate publications*. This
approach allows the plugins to live completely in ignorance
of each other.</div>
<div dir="auto" style="font-family:sans-serif"><br>
</div>
<div dir="auto" style="font-family:sans-serif">The final step
is to associate *both publications to one distribution*,
which composes the publications as they are served.</div>
<div dir="auto" style="font-family:sans-serif"><br>
</div>
<div dir="auto" style="font-family:sans-serif">The downside is
that we have to sync and publish twice, and that the
resulting versions and publications aren't locked together.
But I think this is better than leaving versions and
publications unfinished with the assumption that another
plugin will finish the job. Maybe linking them together
could be a good use of the notes field.</div>
</div>
</div>
</blockquote>
<br>
Pulp should support repositories with composed (mixed) content for
the same reason RH does. The repository is a collection of content
that users want to manage together. Consider the promotion cases:
dev, test, prod.<br>
<br>
<blockquote type="cite"
cite="mid:CA+Kyt=q9Xv5TOs_yCGNvbJHb9zavoSrkY72uqk6-wJ4wJLoX8A@mail.gmail.com">
<div dir="auto">
<div dir="auto">
<div dir="auto" style="font-family:sans-serif"><br>
</div>
</div>
<div dir="auto">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>
<div>
<div><br>
</div>
Depends how we plan this ^ i guess we'll decide
which option 1 or 2 fits better.<br>
<br>
</div>
Don't want to go wild, but what if notion of composed
repos will be so popular in the future that's its
amount will increase? I think we do want to at least
partially being able to sync it and not take the
approach all or nothing?<br>
<br>
</div>
#2 speaks to me more for now.<br>
<div><br>
</div>
</div>
<div class="gmail_extra"><br clear="all">
<div>
<div
class="m_5241349277208944567m_-8498754615663569299gmail_signature"
data-smartmail="gmail_signature">
<div dir="ltr"><br>
<br>
--------<br>
Regards,<br>
<br>
Ina Panova<br>
Software Engineer| Pulp| Red Hat Inc.<br>
<br>
"Do not go where the path may lead,<br>
go instead where there is no path and leave a
trail."<br>
</div>
</div>
</div>
<br>
<div class="gmail_quote">On Mon, May 14, 2018 at 9:44
PM, Jeff Ortel <span dir="ltr"><<a
href="mailto:jortel@redhat.com" rel="noreferrer
noreferrer" target="_blank" moz-do-not-send="true">jortel@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> <font
face="DejaVu Sans">Let's brainstorm on
something.<br>
<br>
Pulp needs to deal with remote repositories that
are composed of multiple content types which may
span the domain of a single plugin. Here are a
few examples. Some Red Hat RPM repositories are
composed of: RPMs, DRPMs, , ISOs and Kickstart
Trees. Some OSTree repositories are composed of
OSTrees & Kickstart Trees. This raises a
question: <br>
<br>
How can pulp3 best support syncing with remote
repositories that are composed of multiple
(unrelated) content types in a way that doesn't
result in plugins duplicating support for
content types?<br>
<br>
Few approaches come to mind:<br>
<br>
1. Multiple plugins (Remotes) participate in the
sync flow to produce a new repository version.</font><font
face="DejaVu Sans"><font face="DejaVu Sans"><br>
2. Multiple plugins (Remotes) are sync'd
successively each producing a new version of a
repository. Only the last version contains
the fully sync'd composition.<br>
3. Plugins share code.</font><br>
4. Other?<br>
<br>
<br>
Option #1: Sync would be orchestrated by core or
the user so that multiple plugins (Remotes)
participate in populating a new repository
version. For example: the RPM plugin (Remote)
and the Kickstart Tree plugin (Remote) would
both be sync'd against the same remote
repository that is composed of both types. The
new repository version would be composed of the
result of both plugin (Remote) syncs. To
support this, we'd need to provide a way for
each plugin to operate seamlessly on the same
(new) repository version. Perhaps something
internal to the RepositoryVersion. The
repository version would not be marked
"complete" until the last plugin (Remote) sync
has succeeded. More complicated than #2 but
results in only creating truly complete versions
or nothing. No idea how this would work with
current REST API whereby plugins provide sync
endpoints.<br>
</font><br>
<font face="DejaVu Sans"><font face="DejaVu Sans">Option
#2: Sync would be orchestrated by core or the
user so that multiple plugins (Remotes) create
successive repository versions. For example:
the RPM plugin (Remote) and the Kickstart Tree
plugin (Remote) would both be sync'd against
the same remote repository that is a
composition including both types. The
intermediate versions would be incomplete. </font></font><font
face="DejaVu Sans"><font face="DejaVu Sans"><font
face="DejaVu Sans"><font face="DejaVu Sans">Only
the last version contains the fully sync'd
composition. This approach can be
supported by core today :) but will
produce incomplete repository versions
that are marked complete=True. This
/seems/ undesirable, right? This may not
be a problem for distribution since I
would imaging that only the last (fully
composed) version would be published. But
what about other usages of the
repository's "latest" version?<br>
<br>
</font></font></font></font><font
face="DejaVu Sans"><font face="DejaVu Sans">Option
#3: requires a plugin to be aware of specific
repository composition(s); other plugins and
creates a code dependency between plugins.
For example, the RPM plugin could delegate
ISOs to the File plugin and Kickstart Trees to
the KickStart Tree plugin.</font><br>
<br>
For all options, plugins (Remotes) need to limit
sync to affect only those content types within
their domain. For example, the RPM (Remote)
sync cannot add/remove ISO or KS Trees.<br>
<br>
I am an advocate of some from of options #1 or
#2. Combining plugins (Remotes) as needed to
deal with arbitrary combinations within remote
repositories seems very powerful; does not
impose complexity on plugin writers; and does
not introduce code dependencies between plugins.<br>
<br>
Thoughts?<br>
</font> </div>
<br>
_______________________________________________<br>
Pulp-dev mailing list<br>
<a href="mailto:Pulp-dev@redhat.com" rel="noreferrer
noreferrer" target="_blank" moz-do-not-send="true">Pulp-dev@redhat.com</a><br>
<a
href="https://www.redhat.com/mailman/listinfo/pulp-dev"
rel="noreferrer noreferrer noreferrer"
target="_blank" moz-do-not-send="true">https://www.redhat.com/mailman/listinfo/pulp-dev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
_______________________________________________<br>
Pulp-dev mailing list<br>
<a href="mailto:Pulp-dev@redhat.com" rel="noreferrer
noreferrer" target="_blank" moz-do-not-send="true">Pulp-dev@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-dev"
rel="noreferrer noreferrer noreferrer" target="_blank"
moz-do-not-send="true">https://www.redhat.com/mailman/listinfo/pulp-dev</a><br>
</blockquote>
</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>