<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p><br>
</p>
<div class="moz-cite-prefix">On 2/19/19 12:30 PM, Brian Bouterse
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAAcvrTHskHE-4_onJJRkEjstGG7efWeDKRwREaxp6px5BzG0ZA@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div>Overall +1, we should make a 1-shot uploader. It will need
a bit of code from plugin writers to make it work though.<br>
</div>
<div><br>
</div>
<div>I believe the "manual and generic" feature offerings we
have now are pretty good. I don't think we should take them
away.<br>
</div>
<div><br>
</div>
<div>To describe yet another related use case; consider the
uploading of a tarball with a lot of units in it (maybe a
whole repo). For instance bunch of Ansible roles in a tarball
is what a user would want to upload to Pulp, so you would want
to upload the whole tarball at once. I see three cases:</div>
<div><br>
</div>
<div>a) The generic method (what we already have). The user
supplies all metadata because we can't know the right
attributes without them telling it to us.</div>
<div>b) The one-shot uploader to create a single content unit
from a single Artifact. I think this is what is being proposed
here. +1 to this.</div>
<div>c) Uploading a bunch of content units in one binary, e.g. a
tarball. We should leave this to plugin writers to implement
for now.</div>
</div>
</blockquote>
<p>Could b & c be the same api? The api could (depending on
what the plugin wants to implement) accept either one unit file or
a tar file with a bunch of different units.<br>
</p>
<blockquote type="cite"
cite="mid:CAAcvrTHskHE-4_onJJRkEjstGG7efWeDKRwREaxp6px5BzG0ZA@mail.gmail.com">
<div dir="ltr">
<div><br>
</div>
<div>In terms of implementing (b) I believe it would be useful
to have a Content unit provide an optional class method that
will instantiate a content unit (but not save it) from an
Artifact. Then core can offer an API that accepts a single
Artifact binary and what content unit type it is and it should
create+save the Artifact and then create+save the Content unit
from that Artifact.</div>
<div><br>
</div>
<div>This doesn't address adding it to/from repositories. That
would be ok too as an additional option, but you probably want
to do it to a whole bunch at once somehow. Also it will need
to go through the tasking system for correctness.<br>
</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Feb 19, 2019 at 11:52
AM David Davis <<a href="mailto:daviddavis@redhat.com"
target="_blank" moz-do-not-send="true">daviddavis@redhat.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">What about the case of multi-artifact content?
Don’t they require separate artifact creation and content
creation routes?<br clear="all">
<div>
<div dir="ltr"
class="gmail-m_-3598769782827170221gmail-m_-2200706327285540451gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div><br>
</div>
<div>David<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Feb 19, 2019 at
11:40 AM Austin Macdonald <<a
href="mailto:austin@redhat.com" target="_blank"
moz-do-not-send="true">austin@redhat.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div>I think the key question to ask is:</div>
<div>What circumstances would require the creation of
Content that would not be met by a one-shot upload?</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Feb 19, 2019
at 11:34 AM Daniel Alley <<a
href="mailto:dalley@redhat.com" target="_blank"
moz-do-not-send="true">dalley@redhat.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px
0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div>@Matthias why would you prefer to keep the
normal create? As you mention, the "orphan
cleanup" issue applies to both, so there's no
advantage given to the former.<br>
</div>
<div><br>
</div>
<div>The normal "create" ( POST
.../content/plugin/type/ ...) is unidiomatic and
inconsistent, because the fields needed to upload
a content are different from the fields on the
actual serializer. Most content types keep
metadata inside the packages themselves, so you
can't let the user specify the content field
values, you have to contort everything so that
instead of hydrating the serializer from user
input, it does so by parsing the content. <br>
</div>
<div><br>
</div>
<div>There's also the issue that the libraries we're
using to parse the (Python and RPM) packages do
some validation on the filename to see that it has
the right extension and so forth, and since
artifacts are content-addressed and don't store
that information, with normal create you have to
pass the filename of the original artifact *back
in* at the time you create it, and then copy the
file from Pulp content storage into a temp
directory under a filename which will validate
properly, which is highly unfortunate. <br>
</div>
<div><br>
</div>
<div>With one-shot upload, you avoid both of those
problems, because there's no broken expectations
as to what fields should be accepted, and because
it should be possible (though I haven't tried it)
to parse the original file *before* saving it as
an artifact, thus avoiding a lot of mess. And you
also get the option to add it straight into a
repository. In my opinion, it's a straight
upgrade.<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Feb 19,
2019 at 10:57 AM Matthias Dellweg <<a
href="mailto:dellweg@atix.de" target="_blank"
moz-do-not-send="true">dellweg@atix.de</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">I have no
objections to having the "one shot upload" or even
"one shot<br>
upload into repositoryversion". I think, i would
like to keep the<br>
'traditional' create anyway.<br>
The problem i see with create and one shot upload
is, that another<br>
thing could have triggered 'delete orphans' at the
wrong time, and you<br>
shiny new content unit disappears, before you can
add it to a<br>
repository version.<br>
<br>
On Mon, 18 Feb 2019 14:41:54 -0500<br>
Austin Macdonald <<a
href="mailto:austin@redhat.com" target="_blank"
moz-do-not-send="true">austin@redhat.com</a>>
wrote:<br>
<br>
> Originally, our upload story was as follows:<br>
> The user will upload a new file to Pulp via
POST to /artifacts/<br>
> (provided by core)<br>
> The user will create a new plugin specific
Content via POST to<br>
> /path/to/plugin/content/, referencing
whatever artifacts that are<br>
> contained, and whatever fields are expected
for the new content.<br>
> The user will add the new content to a
repository via POST to<br>
> /repositories/1/versions/<br>
> <br>
> However, this is somewhat cumbersome to the
user with 3 API calls to<br>
> accomplish something that only took one call
in Pulp 2.<br>
> <br>
> There are a couple of different paths plugins
have taken to improve<br>
> the user experience:<br>
> The Python plugin follows the above workflow,
but reads the Artifact<br>
> file to determine the values for the fields.
The RPM plugin has gone<br>
> even farther and created a new endpoint for
"one shot" upload that<br>
> perform all of this in a single call. I think
it is likely that the<br>
> Python plugin will move more in the "one
shot" direction, and other<br>
> plugins will probably follow.<br>
> <br>
> That said, I think we should discuss this as
a community to encourage<br>
> plugins to behave similarly, and because
there may also be a<br>
> possibility for sharing some of code. It is
my hope that a "one shot<br>
> upload" could do 2 things: 1) Upload and
create Content. 2)<br>
> Optionally add that content to repositories.<br>
_______________________________________________<br>
Pulp-dev mailing list<br>
<a href="mailto:Pulp-dev@redhat.com"
target="_blank" moz-do-not-send="true">Pulp-dev@redhat.com</a><br>
<a
href="https://www.redhat.com/mailman/listinfo/pulp-dev"
rel="noreferrer" target="_blank"
moz-do-not-send="true">https://www.redhat.com/mailman/listinfo/pulp-dev</a><br>
</blockquote>
</div>
</blockquote>
</div>
_______________________________________________<br>
Pulp-dev mailing list<br>
<a href="mailto:Pulp-dev@redhat.com" target="_blank"
moz-do-not-send="true">Pulp-dev@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-dev"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://www.redhat.com/mailman/listinfo/pulp-dev</a><br>
</blockquote>
</div>
_______________________________________________<br>
Pulp-dev mailing list<br>
<a href="mailto:Pulp-dev@redhat.com" target="_blank"
moz-do-not-send="true">Pulp-dev@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-dev"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://www.redhat.com/mailman/listinfo/pulp-dev</a><br>
</blockquote>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Pulp-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Pulp-dev@redhat.com">Pulp-dev@redhat.com</a>
<a class="moz-txt-link-freetext" href="https://www.redhat.com/mailman/listinfo/pulp-dev">https://www.redhat.com/mailman/listinfo/pulp-dev</a>
</pre>
</blockquote>
</body>
</html>