[Pulp-dev] proposing changes to pulp 3 upload API

[Dennis Kliban]

On Tue, Jun 27, 2017 at 1:24 PM, Michael Hrivnak <mhrivnak at redhat.com>
wrote:

>
> On Tue, Jun 27, 2017 at 11:27 AM, Jeff Ortel <jortel at redhat.com> wrote:
>
>>
>> - The artifact FK to a content unit would need to become optional.
>>
>> - Need to add use cases for cleaning up artifacts not associated with a
>> content unit.
>>
>> - The upload API would need additional information needed to create an
>> artifact.  Like relative path, size,
>> checksums etc.
>>
>> - Since (I assume) you are proposing uploading/writing directly to
>> artifact storage (not staging in a working
>> dir), the flow would need to involve (optional) validation.  If
>> validation fails, the artifact must not be
>> inserted into the DB.
>
>
> Perhaps a decent middle ground would be to stick with the plan of keeping
> uploaded (or partially uploaded) files as a separate model until they are
> ready to be turned into a Content instance plus artifacts, and save their
> file data directly to somewhere within /var/lib/pulp/. It would be some
> path distinct from where Artifacts are stored. That's what I had imagined
> we would do anyway. Then as Dennis pointed out, turning that into an
> Artifact would only require a move operation on the same filesystem, which
> is super-cheap.
>
>
Would that address all the concerns? We'd write the data just once, and
> then move it once on the same filesystem. I haven't looked at django's
> support for this recently, but it seems like it should be doable.
>
> I was just looking at the dropbox API and noticed that they provide two
separate API endpoints for regular file uploads[0] (< 150mb) and large file
uploads[1]. It is the latter that supports chunking and requires using an
upload id. For the most common case they support uploading a file with one
API call. Our original proposal requires 2 for the same use case. Pulp API
users would appreciate having to only make one API call to upload a file.

[0] https://www.dropbox.com/developers-v1/core/docs#files_put
[1] https://www.dropbox.com/developers-v1/core/docs#chunked-upload



> --
>
> Michael Hrivnak
>
> Principal Software Engineer, RHCE
>
> Red Hat
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20170627/b0f1d647/attachment.htm>