[Pulp-dev] creating repository version resources using a single REST endpoint

[Bihan Zhang]

On Tue, Jan 9, 2018 at 7:43 AM, Dennis Kliban <dkliban at redhat.com> wrote:

> On Mon, Jan 8, 2018 at 4:24 PM, Austin Macdonald <amacdona at redhat.com>
>  wrote:
>
>> From a discussion with dkliban I see that this design could work. Plugin
>> tasks would be imported to pulpcore with a mechanism similar to the named
>> viewsets and serializers.
>>
>> Pro: plugins would define tasks that follow a consistent interface (sync,
>> rich copy, etc)
>> Con: plugins would be restricted to tasks that are explicitly part of
>> that interface.
>>
>
> I am actually proposing a very loose interface. Any parameters passed in
> the POST body would be dispatched as arguments for the task. The plugin
> author could also define the name they want to use for their task.
>

How would we deal with the order of task execution? In the previous
proposal you mentioned the correct order should be sync -> add -> remove,
if the user can pass any task to this system, would they have any guarantee
of resolution order?

>
>
>> For the docs, I think this puts the endpoint in an awkward position. What
>> does each action do? Would the actions be generic enough that we could
>> correctly explain each of them as part of the core REST API docs?
>>
>> We should be able to dynamically generate help text needed for our REST
> API docs.
>
> We can dynamically generate help text for endpoints, but since the tasks
aren't endpoints I think the user would have to check every plugin doc for
specific task names.

I think this proposal is viable, but, personally, I'm not a fan of having
to check plugin docs for task names, descriptions, parameters, and then
structuring an argument to pass to a centralized endpoint. I would rather
prefer doing one OPTIONS request to a plugin endpoint and seeing the
description/parameters there.

On Tue, Jan 9, 2018 at 7:43 AM, Dennis Kliban <dkliban at redhat.com> wrote:

> On Mon, Jan 8, 2018 at 4:24 PM, Austin Macdonald <amacdona at redhat.com>
> wrote:
>
>> From a discussion with dkliban I see that this design could work. Plugin
>> tasks would be imported to pulpcore with a mechanism similar to the named
>> viewsets and serializers.
>>
>> Pro: plugins would define tasks that follow a consistent interface (sync,
>> rich copy, etc)
>> Con: plugins would be restricted to tasks that are explicitly part of
>> that interface.
>>
>
> I am actually proposing a very loose interface. Any parameters passed in
> the POST body would be dispatched as arguments for the task. The plugin
> author could also define the name they want to use for their task.
>
>
>> For the docs, I think this puts the endpoint in an awkward position. What
>> does each action do? Would the actions be generic enough that we could
>> correctly explain each of them as part of the core REST API docs?
>>
>> We should be able to dynamically generate help text needed for our REST
> API docs.
>
>
>
>> We should also discuss synchronous validation. If a plugin's viewset
>> dispatches their own tasks, they can also define their own POST body
>> requirements aperform arbitrary synchronous validation. If the
>> RepositoryVersionViewset dispatches the task, synchronous validation could
>> still be done as part of the interface, with plugins also defining
>> something like "sync_validation" which would be run before the task is
>> dispatched.
>>
>
> I think the interface for validation could be more generic than that. It
> would be called validate() and would always accept the same parameters as
> the actual task. It's up to the plugin writer to implement it so that the
> REST API can validate input before dispatching a task.
>
>
>
>>
>> Overall, I am convinced that this is a viable option, noting that this
>> design favors consistency between plugins over flexibility. If the plugin
>> viewsets are the ones to dispatch tasks instead, the plugins can do
>> whatever they need to, at the cost of consistency between plugins.
>>
>> On Mon, Jan 8, 2018 at 3:15 PM, Dennis Kliban <dkliban at redhat.com> wrote:
>>
>>> On Mon, Jan 8, 2018 at 2:39 PM, Austin Macdonald <amacdona at redhat.com>
>>> wrote:
>>>
>>>> I like the concept of single REST endpoint that is responsible for all
>>>> the ways to create a RepositoryVersion, but I don't quite understand how
>>>> this would work. Since the endpoint is purely pulpcore, how can the
>>>> RepositoryVersionViewSet import the plugin defined tasks that correspond to
>>>> the action specified by the user? The only way I see is to force plugin
>>>> writers to define all their tasks as methods on the Importer or Publisher,
>>>> which brings us back to the circular import problem.
>>>> https://pulp.plan.io/issues/3074
>>>>
>>>>
>>> Plugin writers would need to define the tasks inside the tasks module of
>>> their django app. pulpcore would then be able to discover the tasks defined
>>> by the plugin at startup. The 'operation' could be name spaced by the
>>> plugin name. Any tasks discovered in pulpcore would have pulpcore prepended
>>> to the operation name. e.g.: pulpcore.sync or pulp_rpm.deep_copy
>>>
>>> This would also address the circular import problem by moving the code
>>> that performs a sync outside the Importer. However, this would require the
>>> plugin writer to instantiate an Importer based on an 'href' passed in as an
>>> argument. And only then could the importer be used to drive the API.
>>>
>>>
>>>> Also, I think it would be a little unusual that the possible actions
>>>> specified in the POST body to a pulpcore endpoint would vary depending on
>>>> the plugin it is being used with. How would we document how to use this
>>>> endpoint?
>>>>
>>>>
>>> The endpoint would have a limited number of operations listed in our
>>> hosted docs. However, the rest API docs on each Pulp installation should be
>>> able to provide the user with a list of all available options.
>>>
>>>
>>>> On Mon, Jan 8, 2018 at 1:45 PM, Dennis Kliban <dkliban at redhat.com>
>>>> wrote:
>>>>
>>>>> Enable users to POST to /api/v3/repositories/123abc456/versions/ with
>>>>> one required parameter 'operation'. This parameter would be an identifier
>>>>> for a task Pulp would run to create a new version. Any additional
>>>>> parameters passed in by the API user would be passed along to the task.
>>>>>
>>>>> pulpcore would provide the 'sync' task and the 'add_remove' task.
>>>>> 'sync' would accept an 'importer'. 'add_remove' would accept
>>>>> 'remove_content' and 'add_content'.
>>>>>
>>>>> Each plugin could provide any number of tasks for creating a
>>>>> repository version.
>>>>>
>>>>> pulpcore would always create the new repository version, hand it to
>>>>> the plugin code, and then mark it as complete after plugin code runs
>>>>> successfully. Alleviating the plugin writer of these concern.
>>>>>
>>>>> REST API users would always use the same end point to create a
>>>>> repository version. Plugin writers wouldn't have to worry about creating
>>>>> repository versions and managing the 'complete' state.
>>>>>
>>>>> What do you all think?
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Pulp-dev mailing list
>>>>> Pulp-dev at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>
>>>>>
>>>>
>>>
>>
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180109/303a520a/attachment.htm>