[Pulp-dev] Changes in the Pulp 3 Upload story

[Daniel Alley]

>
> I think the key question to ask is:
> What circumstances would require the creation of Content that would not be
> met by a one-shot upload?
>

I think that's right.  The answer to this is probably: content with
multiple artifacts, and content with no artifacts.  The latter case doesn't
experience any of the problems we've mentioned, the former case I don't
have any experience with.

But one-shot upload is really just a pattern that fits well for most
plugins, ultimately each plugin can do what works best for their own
scenario.  Plugin writers have their own *ContentViewSets and they can
easily make "normal create" work exactly as it does currently.  Which is to
say that we're not excluding any use cases by making this one use case fit
better.

On Tue, Feb 19, 2019 at 11:40 AM Austin Macdonald <austin at redhat.com> wrote:

> I think the key question to ask is:
> What circumstances would require the creation of Content that would not be
> met by a one-shot upload?
>
> On Tue, Feb 19, 2019 at 11:34 AM Daniel Alley <dalley at redhat.com> wrote:
>
>> @Matthias why would you prefer to keep the normal create?  As you
>> mention, the "orphan cleanup" issue applies to both, so there's no
>> advantage given to the former.
>>
>> The normal "create" ( POST .../content/plugin/type/ ...) is unidiomatic
>> and inconsistent, because the fields needed to upload a content are
>> different from the fields on the actual serializer.  Most content types
>> keep metadata inside the packages themselves, so you can't let the user
>> specify the content field values, you have to contort everything so that
>> instead of hydrating the serializer from user input, it does so by parsing
>> the content.
>>
>> There's also the issue that the libraries we're using to parse the
>> (Python and RPM) packages do some validation on the filename to see that it
>> has the right extension and so forth, and since artifacts are
>> content-addressed and don't store that information, with normal create you
>> have to pass the filename of the original artifact *back in* at the time
>> you create it, and then copy the file from Pulp content storage into a temp
>> directory under a filename which will validate properly, which is highly
>> unfortunate.
>>
>> With one-shot upload, you avoid both of those problems, because there's
>> no broken expectations as to what fields should be accepted, and because it
>> should be possible (though I haven't tried it) to parse the original file
>> *before* saving it as an artifact, thus avoiding a lot of mess. And you
>> also get the option to add it straight into a repository. In my opinion,
>> it's a straight upgrade.
>>
>> On Tue, Feb 19, 2019 at 10:57 AM Matthias Dellweg <dellweg at atix.de>
>> wrote:
>>
>>> I have no objections to having the "one shot upload" or even "one shot
>>> upload into repositoryversion". I think, i would like to keep the
>>> 'traditional' create anyway.
>>> The problem i see with create and one shot upload is, that another
>>> thing could have triggered 'delete orphans' at the wrong time, and you
>>> shiny new content unit disappears, before you can add it to a
>>> repository version.
>>>
>>> On Mon, 18 Feb 2019 14:41:54 -0500
>>> Austin Macdonald <austin at redhat.com> wrote:
>>>
>>> > Originally, our upload story was as follows:
>>> > The user will upload a new file to Pulp via POST to /artifacts/
>>> > (provided by core)
>>> > The user will create a new plugin specific Content via POST to
>>> > /path/to/plugin/content/, referencing whatever artifacts that are
>>> > contained, and whatever fields are expected for the new content.
>>> > The user will add the new content to a repository via POST to
>>> > /repositories/1/versions/
>>> >
>>> > However, this is somewhat cumbersome to the user with 3 API calls to
>>> > accomplish something that only took one call in Pulp 2.
>>> >
>>> > There are a couple of different paths plugins have taken to improve
>>> > the user experience:
>>> > The Python plugin follows the above workflow, but reads the Artifact
>>> > file to determine the values for the fields. The RPM plugin has gone
>>> > even farther and created a new endpoint for "one shot" upload that
>>> > perform all of this in a single call. I think it is likely that the
>>> > Python plugin will move more in the "one shot" direction, and other
>>> > plugins will probably follow.
>>> >
>>> > That said, I think we should discuss this as a community to encourage
>>> > plugins to behave similarly, and because there may also be a
>>> > possibility for sharing some of code. It is my hope that a "one shot
>>> > upload" could do 2 things: 1) Upload and create Content. 2)
>>> > Optionally add that content to repositories.
>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20190219/41bfa397/attachment.htm>