[Pulp-dev] proposing changes to pulp 3 upload API

[Jeff Ortel]

It's my understanding that the advantage of the proposed change to how the storage path is calculated to be
based on the sha256 digest of the artifact's file itself, is so that uploads can be stored directly as
Artifacts.  In other words, an artifact can be created independently of a content unit and stored at a
deterministic location.  This is effectively CAS (content addressed storage).  As a result, each artifact is
absolutely unique and stored exactly once.  In pulp2 and pulp3 (currently), each artifact is unique within its
content unit by relative path.  The binary uniqueness introduces a requirement of a many-to-many relationship
between Artifact <-> Content.  For example: an EL7 RPM that is associated with an RPM content unit and an EL7
distribution.

There is a lot I like about this approach.  However, one thing that needs to be accounted for is the
'relative_path' field on the Artifact.  It represents the artifact's location within the content unit.  This
information is vital to publishing multi-artifact content such as distributions.  Perhaps a modeling change
would resolve this.  Just thinking out loud:

File 1--n Artifact n--1 Content.

File
-----------------
[pk] id
path <--- FileField:  MEDIA_ROOT/files/digest[0:2]/digest[2:]/<name>  # not sure about <name>
downloaded
sha1
sha224
sha256
...


Artifact
------------------
[pk] id
[fk] file_id
[fk] content_id
relative_path

Files can be uploaded directly to `File`.  I think this would support the proposed upload API.

Thoughts?


On 06/30/2017 06:53 AM, Jeff Ortel wrote:
> Perhaps I missed something .. when did the "random path" get proposed?  In pulp2 and in pulp3 (currently), the
> artifact's path is deterministic.
> 
> MEDIA_ROOT/content/units/<type>/digest[0:2]/digest[2:]/<relative-path>
> 
> where The digest is the sha256 hex-digest of the content unit's natural key.  For example, the natural key for
> RPM content is the NEVREA and checksum.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 847 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20170630/61e931ed/attachment.sig>