[Pulp-dev] Namespacing one shot upload and copy endpoints

Daniel Alley dalley at redhat.com
Wed Jul 31 17:13:41 UTC 2019 

I suppose I also worry that doing a switch on "type=" means that adding any
new content type to a plugin means our endpoints would be full of

if type == X:
>
    ...
>
elif type == Y:
>
    ...
>

and about the complexity of that vs the complexity of separate endpoints
once we start gradually more and more advanced features.

Also, if we want a feature for some content types in a plugin but not for
others, if we only have a few heavily overloaded endpoint for copy and
upload, then how do we keep it documented what is supported for type X and
what is not supported?  And if something is not supported then how do we
tell the user this?  The error checking would be pushed down into the
business logic, instead of whether the endpoint accepts a parameter or not.

So, I suppose I prefer this scheme

POST /api/v3/content/rpm/packages/upload/
> POST /api/v3/content/rpm/packages/copy/
>

And pushing as many of the truly generic features to core as possible, such
as copying all units of certain type(s).

On Wed, Jul 31, 2019 at 12:43 PM Daniel Alley <dalley at redhat.com> wrote:

> revision to use case C)
>
> copying all units of a type (e.g. RPM) --> copying all units of one or
> many types (e.g. RPM)
>
> On Wed, Jul 31, 2019 at 12:40 PM Daniel Alley <dalley at redhat.com> wrote:
>
>> For the copy case, it's common to copy more than one type, I think, so
>>> probably 'plugin/copy/ types=[]' makes more sense.
>>>
>>
>> Thanks for this thread. I'm +1 on documenting these general conventions
>>> in the pulpcore-plugin docs for plugin writers. Maybe we could include a
>>> section on URLs so plugin writers could self-assess if they are doing this
>>> or not.
>>>
>>
>> Regarding POST /pulp/api/v3/plugin/action/ types=[]
>>
>> Since this doesn't need to do anything plugin specific (as the type is
>> part of the base content model), my thought was that we would just make it
>> part of core.  Individual plugins would then implement endpoints for more
>> specific behaviors.  I would hope that each plugin wouldn't need to
>> implement the exact same code to copy units based on their type.
>>
>> My thoughts are based on these use cases (I hope I'm not missing
>> anything):
>>
>> A) copying specific content units by content ID
>> B) copying the contents of entire repositories into other repositories
>> C) copying all units of a type (e.g. RPM)  from one repo to another, with
>> no filtering
>> D) copying units of a type (e.g. RPM) from one repo to another if they
>> match some user-provided filters (e.g. name, version)
>> E) copying units of a type (e.g. RPM) from one repo to another if they
>> match some user-provided filters (e.g. name, version), and also copying all
>> their dependencies (using various schemes)
>>
>> Providing a list of types is semi-incompatible with cases D and E
>> (because how would it know how to apply filters to multiple different types
>> at once?  it could probably be done, but it would probably be very messy
>> and very plugin-specific code) so I don't know if that is a good basis for
>> an API that all plugins would build on top of.  On the other hand, A, B,
>> and C can all be accomplished without any plugin involvement -- core could
>> handle those use cases fine, and plugins would be free to handle D and E
>> however they see fit (although we probably do want some common patterns
>> there).
>>
>> But the common patterns for solving D and E are probably going to be
>> different from what we're discussing at the moment.  So I'm not sure if we
>> want to codify POST /pulp/api/v3/plugin/action/ types=[] as our common
>> pattern just yet, at least not without more discussion.
>>
>> On Wed, Jul 31, 2019 at 7:04 AM Tatiana Tereshchenko <ttereshc at redhat.com>
>> wrote:
>>
>>> If the goal is to make endpoints unified across all actions, then I
>>> think we can only do
>>> POST /pulp/api/v3//plugin/action/ types=[]
>>>
>>> Having plugin/content_type/upload would be nice, however I'm not sure if
>>> it covers enough use cases.
>>> E.g. For pulp_rpm,  it makes sense for packages or advisories to have a
>>> dedicated endpoint each, however it doesn't make much sense for modulemd or
>>> modulemd_defaults, because usually they are in the same file and uploaded
>>> in bulk (maybe a separate endpoint is needed for this case).
>>>
>>> For the copy case, it's common to copy more than one type, I think, so
>>> probably 'plugin/copy/ types=[]' makes more sense.
>>>
>>> It would be great to here from more people and other plugins.
>>>
>>>
>>>
>>> On Mon, Jul 29, 2019 at 5:46 PM Pavel Picka <ppicka at redhat.com> wrote:
>>>
>>>> +1 for discuss this to keep some standard as I have already opened PRs
>>>> for rpm modulemd[-defaults].
>>>> I like idea of /upload in the end.
>>>> But also think it can work without as it will be differ by POST/GET
>>>> methods.
>>>>
>>>> On Mon, Jul 29, 2019 at 4:49 PM Dana Walker <dawalker at redhat.com>
>>>> wrote:
>>>>
>>>>> Just to provide an added data point, I'll be merging the one-shot PR
>>>>> for pulp_python soon and it currently uses /api/v3/python/upload/
>>>>>
>>>>> I wanted to keep it simple as well, and so would be happy to change it
>>>>> for consistency based on whatever we decide.
>>>>>
>>>>> --Dana
>>>>>
>>>>> Dana Walker
>>>>>
>>>>> She / Her / Hers
>>>>>
>>>>> Software Engineer, Pulp Project
>>>>>
>>>>> Red Hat <https://www.redhat.com>
>>>>>
>>>>> dawalker at redhat.com
>>>>> <https://www.redhat.com>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Jul 29, 2019 at 10:42 AM Ina Panova <ipanova at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>> As of today, plugins have the freedom to define whichever endpoints
>>>>>> they want ( to some extent).
>>>>>> This leads to the question - shall we namespace one-shot upload and
>>>>>> copy endpoints for some consistency?
>>>>>>
>>>>>> POST /api/v3/content/rpm/packages/upload/
>>>>>> POST /api/v3/content/rpm/packages/copy/
>>>>>>
>>>>>> or
>>>>>>
>>>>>> POST /api/v3/content/rpm/upload/ type =package
>>>>>> POST /api/v3/content/rpm/copy/ type = [package, modulemd]
>>>>>>
>>>>>> I wanted to bring this up, before it diverges a lot. For the record,
>>>>>> I have checked only RPM plugin, I am not aware of the state of the other
>>>>>> plugins.
>>>>>> Right now we have an active endpoint for one-shot upload of rpm
>>>>>> package:
>>>>>> POST /api/v3/content/rpm/upload/
>>>>>>
>>>>>> And there is PR for one-shot upload of modulemd-defaults:
>>>>>> POST /api/v3/content/rpm/modulemd-defaults/
>>>>>>
>>>>>> For rpm copy we have POST /api/v3/content/rpm/copy/ types=[]
>>>>>>
>>>>>> We are starting some work on docker recursive copy, so it would be
>>>>>> helpful to reach some agreement before going further that path.
>>>>>>
>>>>>> Thank you!
>>>>>> --------
>>>>>> Regards,
>>>>>>
>>>>>> Ina Panova
>>>>>> Senior Software Engineer| Pulp| Red Hat Inc.
>>>>>>
>>>>>> "Do not go where the path may lead,
>>>>>>  go instead where there is no path and leave a trail."
>>>>>> _______________________________________________
>>>>>> Pulp-dev mailing list
>>>>>> Pulp-dev at redhat.com
>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>
>>>>> _______________________________________________
>>>>> Pulp-dev mailing list
>>>>> Pulp-dev at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>
>>>>
>>>>
>>>> --
>>>> Pavel Picka
>>>> Red Hat
>>>> _______________________________________________
>>>> Pulp-dev mailing list
>>>> Pulp-dev at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>
>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20190731/1b375bd9/attachment.htm>

More information about the Pulp-dev mailing list