[Pulp-dev] Pulp 3 content import

Austin Macdonald austin at redhat.com
Wed Apr 18 16:46:06 UTC 2018


On Wed, Apr 18, 2018 at 12:03 PM, Dennis Kliban <dkliban at redhat.com> wrote:

> On Wed, Apr 18, 2018 at 11:02 AM, Simon Baatz <gmbnomis at gmail.com> wrote:
>
>> I am looking at Pulp 3 for the first time and have a question on
>> creating/importing content units.
>>
>> In the file plugin, a file content unit can be created using
>> POST at .../content/file/ by passing the meta-data ('relative_path') and the
>> artifact (from which 'digest' is obtained as additional meta-data).  The
>> POST is synchronous (returning a 201 CREATED if successful).
>>
>> However, for other artifact types there is more meta-data in the
>> artifact(s)
>> themselves and it may be "harder" to obtain.  For example, Puppet modules,
>> Chef cookbooks, and Helm charts are tarballs with specific meta-data
>> files.
>> In Pulp 2, this meta-data extraction seems to be part of the
>> import_uploaded_unit task.  According to the code comments, the puppet
>> module unpacks the entire module to get the meta-data.
>>
>> If I got this correctly, Pulp 3 splits this up into content unit
>> creation (sync) and "importing" (async, add content units using
>> POST at ...versions/).  Where should the meta-data extraction be done
>> given this split?  (the first step is synchronous, but perhaps it is
>> acceptable to do some work there nevertheless. The second step
>> looks too late)
>>
>>
> The REST API provided by pulpcore in Pulp 3 is very generic. As you
> pointed out, it requires the REST API client to know all the metadata
> required to create a piece of Content from an Artifact. So I picture this
> process as a 3 step process in the client:
>
> 1) Inspect the artifact to extract all needed metadata
> 2) Use Artifact creation REST API to upload Artifact to Pulp
> 3) Use SomeContent creation REST API to create SomeContent from the
> uploaded Artifact and the metadata extracted in step 1
>

"smart upload" doesn't necessarily have to be a new endpoint.

1) POST /artifacts/
    user uploads artifact (just the tarball, no metadata needed)

2) POST /content/SomeUnit/
   user creates SomeUnit, and specifies the artifacts.  This is a plugin
implemented endpoint already, and I think it makes sense to crack up the
tarballs and create the metadata here. The call is currently synchronous
though, we might want to make that async.

https://github.com/pulp/pulp_file/blob/master/pulp_file/app/viewsets.py#L71


> With all that said, each plugin is also be able to provide additional REST
> API endpoints. I suspect that some of these endpoints will support "smart
> upload" that will be aware of the content type that is being uploaded.
> These endpoints will allow users to upload an Artifact of a specific type
> and the server will extract all the needed metadata to create the
> SomeContent in Pulp.
>
>
>> - Simon
>>
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180418/5911e794/attachment.htm>


More information about the Pulp-dev mailing list