[Pulp-dev] Changes in the Pulp 3 Upload story

Tue Feb 19 16:59:26 UTC 2019

If you parse all the metadata from the file / existing artifact, i see
no real difference (from the users perspective). One call would accept
a file upload, the other a reference to an existing artifact. The
latter might impose a problem internally if the artifacts are stored
*somewhere*, as you say. In the case of the file plugin, you need to
specify a file / artifact_ref and a relative_path. So no big difference
again.
What i do not see is, how it is supposed to work for
multiple-artifact content (does it really exist? Docker?). Can one
upload multiple files in one api-call?

Does the internal copying around hurt so much as to not wanting to
implement the create endpoint?

I know, the orphaning problem is really only circumvented by an atomic
one shot into the repo version endpoint. But i thought the problem is
related to this story.

On Tue, 19 Feb 2019 11:33:18 -0500
Daniel Alley <dalley at redhat.com> wrote:

> @Matthias why would you prefer to keep the normal create?  As you
> mention, the "orphan cleanup" issue applies to both, so there's no
> advantage given to the former.
> 
> The normal "create" ( POST .../content/plugin/type/ ...) is
> unidiomatic and inconsistent, because the fields needed to upload a
> content are different from the fields on the actual serializer.  Most
> content types keep metadata inside the packages themselves, so you
> can't let the user specify the content field values, you have to
> contort everything so that instead of hydrating the serializer from
> user input, it does so by parsing the content.
> 
> There's also the issue that the libraries we're using to parse the
> (Python and RPM) packages do some validation on the filename to see
> that it has the right extension and so forth, and since artifacts are
> content-addressed and don't store that information, with normal
> create you have to pass the filename of the original artifact *back
> in* at the time you create it, and then copy the file from Pulp
> content storage into a temp directory under a filename which will
> validate properly, which is highly unfortunate.
> 
> With one-shot upload, you avoid both of those problems, because
> there's no broken expectations as to what fields should be accepted,
> and because it should be possible (though I haven't tried it) to
> parse the original file *before* saving it as an artifact, thus
> avoiding a lot of mess. And you also get the option to add it
> straight into a repository. In my opinion, it's a straight upgrade.
> 
> On Tue, Feb 19, 2019 at 10:57 AM Matthias Dellweg <dellweg at atix.de>
> wrote:
> 
> > I have no objections to having the "one shot upload" or even "one
> > shot upload into repositoryversion". I think, i would like to keep
> > the 'traditional' create anyway.
> > The problem i see with create and one shot upload is, that another
> > thing could have triggered 'delete orphans' at the wrong time, and
> > you shiny new content unit disappears, before you can add it to a
> > repository version.
> >
> > On Mon, 18 Feb 2019 14:41:54 -0500
> > Austin Macdonald <austin at redhat.com> wrote:
> >  
> > > Originally, our upload story was as follows:
> > > The user will upload a new file to Pulp via POST to /artifacts/
> > > (provided by core)
> > > The user will create a new plugin specific Content via POST to
> > > /path/to/plugin/content/, referencing whatever artifacts that are
> > > contained, and whatever fields are expected for the new content.
> > > The user will add the new content to a repository via POST to
> > > /repositories/1/versions/
> > >
> > > However, this is somewhat cumbersome to the user with 3 API calls
> > > to accomplish something that only took one call in Pulp 2.
> > >
> > > There are a couple of different paths plugins have taken to
> > > improve the user experience:
> > > The Python plugin follows the above workflow, but reads the
> > > Artifact file to determine the values for the fields. The RPM
> > > plugin has gone even farther and created a new endpoint for "one
> > > shot" upload that perform all of this in a single call. I think
> > > it is likely that the Python plugin will move more in the "one
> > > shot" direction, and other plugins will probably follow.
> > >
> > > That said, I think we should discuss this as a community to
> > > encourage plugins to behave similarly, and because there may also
> > > be a possibility for sharing some of code. It is my hope that a
> > > "one shot upload" could do 2 things: 1) Upload and create
> > > Content. 2) Optionally add that content to repositories.  
> > _______________________________________________
> > Pulp-dev mailing list
> > Pulp-dev at redhat.com
> > https://www.redhat.com/mailman/listinfo/pulp-dev
> >  

Herzliche Grüße aus München

Matthias Dellweg
______________________________________________________ 
Dr. Matthias M. Dellweg

(Open Source Software Engineer)

Tel: +49 (0)89 452 35 38-12
Fax: +49 (0)89 452 35 38-290
E-Mail: dellweg at atix.de

ATIX - The Linux & Open Source Company

ATIX Informationstechnologie und Consulting AG
Parkring 15
85748 Garching bei München
www.atix.de

Registergericht: Amtsgericht München, Registernummer: HRB 168930
USt.-Id.: DE209485962
Vorstand: Thomas Merz (Vors.), Mark Hlawatschek
Vorsitzender des Aufsichtsrats: Dr. Martin Buss
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20190219/22ff9e61/attachment.sig>