<div dir="ltr">+1 to option 2.<div class="gmail_extra"><div><div class="m_-4709153293767703112gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><br></div><div>David<br></div></div></div></div></div></div></div></div>
<br><div class="gmail_quote">On Tue, Apr 17, 2018 at 9:39 AM, Brian Bouterse <span dir="ltr"><<a href="mailto:bbouters@redhat.com" target="_blank">bbouters@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>I believe option 2 would be a good improvement for Pulp's API. It would resolve the current problem that a RepoVersion and/or a Publication can be created in multiple places in the REST API.<br></div></div><div class="m_-4709153293767703112HOEnZb"><div class="m_-4709153293767703112h5"><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Apr 13, 2018 at 3:55 PM, Jeff Ortel <span dir="ltr"><<a href="mailto:jortel@redhat.com" target="_blank">jortel@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><div><div class="m_-4709153293767703112m_9074744428784529669h5">
<br>
<br>
<div class="m_-4709153293767703112m_9074744428784529669m_-5698888641570368801moz-cite-prefix">On 04/12/2018 04:49 PM, Dennis Kliban
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Thu, Apr 12, 2018 at 2:49 PM, Jeff
Ortel <span dir="ltr"><<a href="mailto:jortel@redhat.com" target="_blank">jortel@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>
<div class="m_-4709153293767703112m_9074744428784529669m_-5698888641570368801m_-8082973026962369681m_2359146138886405596m_-6975650155196327159h5">
<br>
<br>
<div class="m_-4709153293767703112m_9074744428784529669m_-5698888641570368801m_-8082973026962369681m_2359146138886405596m_-6975650155196327159m_7479694530690954398moz-cite-prefix">On
04/11/2018 01:13 PM, Dennis Kliban wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Tue, Apr 10, 2018
at 6:44 PM, Jeff Ortel <span dir="ltr"><<a href="mailto:jortel@redhat.com" target="_blank">jortel@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><span>
<br>
<br>
<div class="m_-4709153293767703112m_9074744428784529669m_-5698888641570368801m_-8082973026962369681m_2359146138886405596m_-6975650155196327159m_7479694530690954398m_8733354028057534058m_-4789151742247051920moz-cite-prefix">On
04/10/2018 04:15 PM, Dennis Kliban
wrote:<br>
</div>
</span>
<div>
<div class="m_-4709153293767703112m_9074744428784529669m_-5698888641570368801m_-8082973026962369681m_2359146138886405596m_-6975650155196327159m_7479694530690954398m_8733354028057534058h5">
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On
Tue, Apr 10, 2018 at 2:04
PM, Brian Bouterse <span dir="ltr"><<a href="mailto:bbouters@redhat.com" target="_blank">bbouters@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>These are good
problem statements. I
didn't understand all
of the aspects of it,
so I put some inline
questions.<br>
<br>
My overall question
is: are these related
problems? To share my
answer to this, I
believe the first two
problems are related
and the third is
separate. The classic
divide and conquor
approach we could use
here is to confirm
that the problems are
unrelated and focus on
resolving one of them
first.<br>
<br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>I don't think all 3 are
related problems. The
motivation for grouping
all together is that a
subset of the action
endpoints from problem 1
are used to create
repository versions and
Problem 3 is a problem
with the repository
version creation API. <br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div class="gmail_extra"><br>
<div class="gmail_quote"><span>On
Mon, Apr 9,
2018 at 3:18
PM, Austin
Macdonald <span dir="ltr"><<a href="mailto:austin@redhat.com" target="_blank">austin@redhat.com</a>></span>
wrote:<br>
</span><span>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>Folks,</div>
<div><br>
</div>
<div>Austin,
Dennis, and
Milan have
identified the
following
issues with
current Pulp3
REST API
design:</div>
<div>
<ul>
<li>Action
endpoints are
problematic.<br>
</li>
<ul>
<li>Example
POST@/importers/<plugin>/sync/</li>
<li>They are
non-RESTful
and would make
client code
tightly
coupled with
the server
code.</li>
<li>These
endpoints are
inconsistent
with the other
parts of the
REST API.</li>
</ul>
</ul>
</div>
</div>
</blockquote>
</span>
<div>Is
self-consistency
really a goal?
I think it's a
placeholder
for
consistency
for REST since
the "rest" of
the API is
RESTful. After
reading parts
of Roy
Fielding's
writeup of the
definition of
REST I believe
"action
endpoints are
not RESTful"
to be a true
statement.
Maybe "Action
endpoints are
problematic"
should be
replaced with
"Action
endpoints are
not RESTful"
perhaps and
have the
self-consistency
bullet
removed?<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>+1 to "Action endpoints
are not RESTful"<br>
</div>
<div>+1 to removing the
self-consistency language<br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div class="gmail_extra">
<div class="gmail_quote"><span>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>
<ul>
<ul>
<li>DRF is not
being used as
intended for
action
endpoints so
we have to
implement
extra code.
(against the
grain)</li>
</ul>
</ul>
</div>
</div>
</blockquote>
</span>
<div>I don't
know much
about this.
Where is the
extra code?<br>
</div>
<span>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>
<ul>
<li>We don't
have a
convention for
where
plug-in-specific,
custom
repository
version
creation
endpoints.<br>
</li>
<ul>
<li>example
POST@/api/v3/<where?>/docker/a<wbr>dd/</li>
<li>needs to
be
discoverable
through the
schema</li>
</ul>
</ul>
</div>
</div>
</blockquote>
</span>
<div>What does
discoverable
via the schema
^ mean? Aren't
all urls
listed in the
schema?<br>
<br>
</div>
<div>I think
of ^ problem
somewhat
differently.
Yes all urls
need to be
discoverable
(a REST
property), but
isn't it more
of an issue
that the urls
which produce
repo versions
can't be
identified
distinctly
from any other
plugin-contributed url? To paraphrase this perspective: making a repo
version is
strewn about
throughout the
API in random
places which
is a bad user
experience. Is
that what is
motivation url
discovery?<br>
</div>
<span>
<div> </div>
</span></div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>Yes. I envision a CLI
that can discover new
plugin
repository-version-creating
functionality without
having to install new
client packages. Allowing
plugin writers to add
endpoints in arbitrary
places for creating
repository versions will
make it impossible for the
client to know what all
the possible ways of
creating a repository
version are. <br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div class="gmail_extra">
<div class="gmail_quote"><span>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>
<ul>
<li>For direct
repository
version
creation,
plugins are
not involved.<br>
</li>
<ul>
<li>validation
correctness
problem: <a href="https://pulp.plan.io/issues/3541" target="_blank">https://pulp.plan.io/issues/35<wbr>41</a></li>
<li>example:
POST@/api/v3/repositories/<rep<wbr>ository_id>/versions/</li>
</ul>
</ul>
</div>
</div>
</blockquote>
</span>
<div>I agree
with this
problem
statement. In
terms of scope
it affects
some plugin
writers but
not all.<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>I think it affects all
plugin writers. Even the
File plugin needs to
provide some validation
when creating a repository
version. Right now you can
add a FileContent with the
same relative path as
another FileContent in the
repository version. This
should not be possible
because it's not a valid
combination of FileContent
units in the same
repository version. <br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</div>
Not necessarily. Two files with the
same relative path will have different
digest (different content). The
assumption that they both cannot be in
the same repository makes assumptions
about how the repository is used which I
don't think is a good idea. Image two
different versions of abc.iso.<span><br>
<br>
</span></div>
</blockquote>
<div><br>
</div>
<div>Why is it bad to assume that a
repository version is going to be
published? What are the other ways to use
a repository version?<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</div>
</div>
The repository may not be publish and/or may not be
published by the FilePublisher in the file plugin
project. A user may want to sync and store many version
of an iso in the repository and then selectively <i>add</i>
a specific version to another repository for promotion
work flows. Also, the user could use another (custom)
publisher that deals differently with multiple files
with the same path in the repository. The FilePublisher
currently publishes the newest. My point being, we,
really cannot assume how the repository will be used or
which publisher <i>may</i> publish it.<span><br>
<br>
</span></div>
</blockquote>
<div><br>
</div>
<div>The problem was initially stated as "For direct
repository version creation, plugins are not involved". It
sounds like you disagree that this is a problem. </div>
</div>
</div>
</div>
</blockquote>
<br></div></div>
Yes. Definitely, agreed.<span><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>Can you confirm this by telling us if plugins should be
able to provide validation for this API provided by core?<br>
</div>
</div>
</div>
</div>
</blockquote>
<br></span>
Plugins participating in core endpoints is different, broader
discussion. <br>
<br>
The following is not aimed at you dkliban :)<br>
<br>
We need to decide if we want to return to the pulp2 pattern whereby
the core delegates behavior to plugins via the plugin API. Or,
continue down the pulp3 path whereby operations involving plugins
are contributed to the API by each plugin (not making a value
judgment). Also, I value consistency in APIs and don't think these
approaches should be mixed (with the exception of content related
live-API). Consistency in APIs reflect both a thoughtful, mature
design and provides a better user experience. I'm sure everyone has
cursed APIs that did things every-which-way. I don't think there is
any difference between creating a repository version via sync or
creating a version with a list of content to add/remove. And to a
lesser degree publishing. We should either POST to the
/publications/ endpoint for creating a publication (core API), <u>or</u>
users should POST to the plugin contributed endpoint (as currently)
for publishing.<br>
<br>
Seems to me, there are 2 high-level choices:<br>
<br>
<u>1. Core endpoints do not delegate/redirect to plugins.</u><br>
- POST to /RepositoryVersion/ is removed.<br>
- POST to /Publications/ (stays gone)<br>
- Plugins provide endpoints for sync and other to create new
repository versions.<br>
- Plugins provide endpoints for creating Publications
(publishing).<br>
<br>
<u>2. Core delegates behavior to plugins for those endpoints
requiring plugin participation.</u><br>
- POST to /RepositoryVersion/ is the only way to create a
repository version.<br>
- POST to /Publications/ is the only way to create a
Publication (publish). <br>
- The POST parameters or body includes enough information so
that core can select a plugin.<br>
- Either the entire POST is passed along to the plugin, <u>or</u>
the plugin implements an API that's used by<br>
core for pre-defined participation.<br>
<br>
There have been proposals on how both #1 and #2 can be achieved. 'm
wondering if we can even agree on one of these two approaches.<div><div class="m_-4709153293767703112m_9074744428784529669h5"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><span>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>A File repository version cannot be
properly published if it contains 2
FileContent units that both have the same
relative path. The publisher has to decide
which FileContent to publish at the relative
path. That decision cannot be made
intelligently by the publisher. The decision
on which content unit to include in the
repository version rests with the user that
is creating the repository version. When a
user tries to create a repository version
with a FileContent that has the same
relative path as another FileContent that
was added previously, Pulp needs to inform
the user that there is a conflict (and not
create the repositiory version). This
validation can only be provided by the File
plugin. <br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><span>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div class="gmail_extra">
<div class="gmail_quote">
<div> <br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span>
<div dir="ltr">
<div>We would
like to get
feedback on
these issues
being sound
and worth
resolving
before we
resume
particular
solution
discussion[1].</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Austin,
Dennis, and
Milan</div>
<div><br>
</div>
<div>[1] <a href="https://www.redhat.com/archives/pulp-dev/2018-March/msg00066.html" target="_blank">https://www.redhat.com/archive<wbr>s/pulp-dev/2018-March/msg00066<wbr>.html</a><br>
</div>
<div><br>
</div>
</div>
<br>
</span><span>______________________________<wbr>_________________<br>
Pulp-dev
mailing list<br>
<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman<wbr>/listinfo/pulp-dev</a><br>
<br>
</span></blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
______________________________<wbr>_________________<br>
Pulp-dev mailing list<br>
<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman<wbr>/listinfo/pulp-dev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
<br>
<fieldset class="m_-4709153293767703112m_9074744428784529669m_-5698888641570368801m_-8082973026962369681m_2359146138886405596m_-6975650155196327159m_7479694530690954398m_8733354028057534058m_-4789151742247051920mimeAttachmentHeader"></fieldset>
<br>
<pre>______________________________<wbr>_________________
Pulp-dev mailing list
<a class="m_-4709153293767703112m_9074744428784529669m_-5698888641570368801m_-8082973026962369681m_2359146138886405596m_-6975650155196327159m_7479694530690954398m_8733354028057534058m_-4789151742247051920moz-txt-link-abbreviated" href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a>
<a class="m_-4709153293767703112m_9074744428784529669m_-5698888641570368801m_-8082973026962369681m_2359146138886405596m_-6975650155196327159m_7479694530690954398m_8733354028057534058m_-4789151742247051920moz-txt-link-freetext" href="https://www.redhat.com/mailman/listinfo/pulp-dev" target="_blank">https://www.redhat.com/mailman<wbr>/listinfo/pulp-dev</a>
</pre>
</blockquote>
<br>
</span></div>
<br>
______________________________<wbr>_________________<br>
Pulp-dev mailing list<br>
<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman<wbr>/listinfo/pulp-dev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
</span></div>
<br>
______________________________<wbr>_________________<br>
Pulp-dev mailing list<br>
<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman<wbr>/listinfo/pulp-dev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
</div></div></div>
<br>______________________________<wbr>_________________<br>
Pulp-dev mailing list<br>
<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman<wbr>/listinfo/pulp-dev</a><br>
<br></blockquote></div><br></div>
</div></div><br>______________________________<wbr>_________________<br>
Pulp-dev mailing list<br>
<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman<wbr>/listinfo/pulp-dev</a><br>
<br></blockquote></div><br></div></div>