[Pulp-dev] proposing changes to pulp 3 upload API

[Dennis Kliban]

I just submitted a PR[0] for uploading Artifacts. The API supports creating
Artifacts, listing all Artifacts, and viewing details of an Artifact.
Updates of Artifacts via REST API are disabled. On upload of a file, all
possible checksums are calculated and compared to any checksums provided by
the user. Any inconsistencies are raised as validation errors.

I am now starting to work on the story[1] #2872 for creating Content units
from Artifacts.


[0] https://github.com/pulp/pulp/pull/3080
[1] https://pulp.plan.io/issues/2872

On Thu, Jul 6, 2017 at 6:23 PM, Dennis Kliban <dkliban at redhat.com> wrote:

> I added 2 more stories based on the discussion we've had so far. Please
> provide feedback here or on the tickets.
>
> https://pulp.plan.io/issues/2872
> https://pulp.plan.io/issues/2873
>
> On Fri, Jun 30, 2017 at 2:23 PM, Brian Bouterse <bbouters at redhat.com>
> wrote:
>
>> @jortel I think what you've written is what we should do. I think we can
>> get a race-condition free implementation with this many-to-many table with
>> the database transaction including the the filesystem operations. +1 to
>> adding the relative path to the join table also.
>>
>> I'm also not sure about including <name> in File.path <--- FileField:
>> MEDIA_ROOT/files/digest[0:2]/digest[2:]/<name>  If there were
>> collisions, having <name> would allow you to store two Files with different
>> contents but the same hash. It's highly, highly improbable even if Pulp is
>> storing billions of files, but it could happen. A content addressable store
>> (not including <name>) would never be able to handle this case.
>>
>> Even still, I think we should disinclude the name from the path to the
>> Artifact and just have it be the digest. Having a fully addressable File
>> storage will be awesome, and a bit more complex code in core is worth it (I
>> think). FWIW, I also don't think it will be that hard to get right.
>>
>> As a side point, I think when a file with the same sha256 is uploaded a
>> second time it should be rejected rather than silently accepting it.
>>
>> On Fri, Jun 30, 2017 at 12:00 PM, Jeff Ortel <jortel at redhat.com> wrote:
>>
>>> Ah, I missed adding the relative path to the join table.  This is a fine
>>> idea as well.
>>>
>>> On 06/30/2017 10:15 AM, Michael Hrivnak wrote:
>>> >
>>> > Jeff, earlier in the thread we talked about using the through table to
>>> hold the path. I think that's the right
>>> > place, because the path would be a property of the relationship
>>> between an artifact and a content unit. It
>>> > also occurred to me that the file name could be different for
>>> different content, so maybe the path would need
>>> > to include the filename. That seems a bit weird, but I think it has to
>>> be the case if we use a many-to-many
>>> > relationship.
>>>
>>>
>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>>
>>
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20170712/5bb1b8a9/attachment.htm>