<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Feb 22, 2019 at 9:36 AM Justin Sherrill <<a href="mailto:jsherril@redhat.com">jsherril@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p><br>
</p>
<div class="gmail-m_1484403417921773583moz-cite-prefix">On 2/18/19 2:41 PM, Austin Macdonald
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Originally, our upload story was as follows:</div>
<div>The user will upload a new file to Pulp via POST to
/artifacts/ (provided by core)<br>
</div>
<div>The user will create a new plugin specific Content via POST
to /path/to/plugin/content/, referencing whatever artifacts
that are contained, and whatever fields are expected for the
new content.<br>
</div>
<div>The user will add the new content to a repository via POST
to /repositories/1/versions/<br>
</div>
<div><br>
</div>
<div>However, this is somewhat cumbersome to the user with 3 API
calls to accomplish something that only took one call in Pulp
2. <br>
</div>
</div>
</blockquote>
<p>How would you do this with one call in pulp2?
<a class="gmail-m_1484403417921773583moz-txt-link-freetext" href="https://docs.pulpproject.org/dev-guide/integration/rest-api/content/upload.html" target="_blank">https://docs.pulpproject.org/dev-guide/integration/rest-api/content/upload.html</a>
seems to suggest 3-4 calls.<br></p></div></blockquote><div>Some plugins implemented the pulp2 equivalent of a one-shot uploader. Those docs are for pulp2's core which don't include the plugin's docs. <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF"><p>
</p>
<blockquote type="cite">
<div dir="ltr">
<div><br>
</div>
<div>There are a couple of different paths plugins have taken to
improve the user experience:</div>
<div>The Python plugin follows the above workflow, but reads the
Artifact file to determine the values for the fields. The RPM
plugin has gone even farther and created a new endpoint for
"one shot" upload that perform all of this in a single call. I
think it is likely that the Python plugin will move more in
the "one shot" direction, and other plugins will probably
follow.<br>
</div>
</div>
</blockquote>
<p>How does the RPM one shot api work? Will it be compatible with
whatever solution <a class="gmail-m_1484403417921773583moz-txt-link-freetext" href="https://pulp.plan.io/issues/4196" target="_blank">https://pulp.plan.io/issues/4196</a> arrives at? </p></div></blockquote><div>You would upload the Artifact as binary data along with what content type it is and what relative path it uses and Pulp creates the Artifact, Content unit, ContentArtifact. It should be compatible with issue 4196 because django's binary form data should allow for parallel uploading before calling the view handler. It may take 2 calls though. The issue to me isn't about the number of calls as it is the client data payload complexity. <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF">
<p>I would hate for all our plugins to move to One shot methods
which users can't even rely on.<br></p></div></blockquote><div>I don't think we're taking the "generic" uploading away. You can always rely on that. The issue w/ one-shot is that it's not possible (literally) for many content types, e.g. Artifact-less content. It's also hard for multi-artifact Content so that would probably still be something plugin writers would provide as a custom thing for their content type. Regardless it's just not possible to have consistency in this area. <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF"><p>
</p>
<p>My problem with single api calls to upload files is that we
cannot reliably use them due to limitation in request sizes. We
have to be prepared to use multiple calls to upload files
regardless. Maybe if a user is using some plugin that never has
super large files (ansible?) you could be confident you would
never hit a request size limitation. But file, docker, and yum
all would require multiple calls to get the physical data to the
server.<br></p></div></blockquote><div>I believe arbitrarily large files can be uploaded either through multi-part form data or through the django-chunked interface. We'll see what happens with 4196, but I expect arbitrary payload size to be a requirement for Pulp users.<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF"><p>
</p>
<p>I care more about having a consistent method for uploading files
than having fewer api calls. If we need a some content specific
api, that's fine, but please make it a consistent part of the
process. <br></p></div></blockquote><div>It sounds like the 4-call interface is the only choice then if consistency is a must. There isn't a way to offer consistency for one-shot uploaders. Is it ok that Katello will have to fill out all of the field data when you post the content type? What could be better?<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div bgcolor="#FFFFFF"><p>
</p>
<p>I feel like we may be chasing the wrong goal here (fewer calls vs
a more consistent experience). <br>
</p>
<blockquote type="cite">
<div dir="ltr">
<div><br>
</div>
<div>That said, I think we should discuss this as a community to
encourage plugins to behave similarly, and because there may
also be a possibility for sharing some of code. It is my hope
that a "one shot upload" could do 2 things: 1) Upload and
create Content. 2) Optionally add that content to
repositories.</div>
</div>
<br>
<fieldset class="gmail-m_1484403417921773583mimeAttachmentHeader"></fieldset>
<pre class="gmail-m_1484403417921773583moz-quote-pre">_______________________________________________
Pulp-dev mailing list
<a class="gmail-m_1484403417921773583moz-txt-link-abbreviated" href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a>
<a class="gmail-m_1484403417921773583moz-txt-link-freetext" href="https://www.redhat.com/mailman/listinfo/pulp-dev" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>
</pre>
</blockquote>
</div>
_______________________________________________<br>
Pulp-dev mailing list<br>
<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a><br>
</blockquote></div></div>