[Pulp-dev] Plugin relationship to tasks

Milan Kovacik mkovacik at redhat.com
Tue Mar 27 07:30:03 UTC 2018


On Tue, Mar 27, 2018 at 12:00 AM, Brian Bouterse <bbouters at redhat.com> wrote:
> I also want to look at how this will affect our user experience. With this
> change, I believe plugin tasks are put at the center of the user experience
> and the repositories, content, and versions are moved to the background. The
> user would think about what they can do foremost as a series of tasks. This
> is great in terms of organization, but I'm concerned its stepping away from
> what makes pulp special. I think Pulp is special because of its focus on
> content management (repos, version, uploads, sync, promotion, copy/delete,
> publish, export).
Fwiw you just enumerated 7 verbs vs 2 nouns ;)

> Having the user think about Pulp as a task-running system
> takes Pulp a step closer to a software like RunDeck
> (https://www.rundeck.com/open-source).
>
> What I like about the existing user experience (and API) is that it all
> centers around Pulp's value proposition (importers, syncs, associations,
> etc).
The Pulp focus isn't repo/version/content/ objects CRUD.
The value added are those uploads, syncs, promotions, copies,
publishes and exports.

If we don't like 'tasks' we can have a poll about a better name of the
endpoint; 'actions' anyone? Or perhaps 'deeds'?

>
> Does this spark any thoughts or ideas from others?
>
>
> On Mon, Mar 26, 2018 at 4:43 PM, Dennis Kliban <dkliban at redhat.com> wrote:
>>
>> On Mon, Mar 26, 2018 at 3:38 PM, Austin Macdonald <austin at redhat.com>
>> wrote:
>>>
>>>
>>>
>>> On Mon, Mar 26, 2018 at 2:30 PM, Dennis Kliban <dkliban at redhat.com>
>>> wrote:
>>>>
>>>> This proposal does not make the plugin writer's job any easier. This
>>>> simply changes where the plugin writer is providing validation logic, in a
>>>> serializer vs. in a view.
>>>
>>>
>>> It does make the plugin writer's job easier *because* it changes how the
>>> validation is provided, Plugin writers already have to understand how to
>>> create models/serializers (and it is very simple) for all the other objects.
>>> Seriallizer validation is much simpler than viewset validation because
>>> serializers are literally designed to do this.
>>
>>
>> If we don't change anything, a plugin writer is responsible for creating a
>> task that creates a Repository Version and a view which will perform
>> validation and then dispatches a task. If we do make the proposed change,
>> the plugin writer will still have to write the task code and then also a
>> serializer that performs validation. The proposed change only removes the
>> responsibility of dispatching a task which is 1 line of code.
>>
>>>
>>>
>>>>
>>>>
>>>> The other problem I see is that complexity for users is increased.
>>>> Instead of having 1 resource for tracking task progress, there would be an
>>>> infinite number of resources for tracking progress of tasks.
>>>
>>>
>>> In the proposal, Tasks are Master/Detail. The user doesn't have to
>>> "track" it at all. They can still use v3/tasks/ to see all the tasks.
>>> Retrieving tasks will either be the same (v3/tasks/) or easier if they know
>>> the task they are looking for (v3/tasks/file/syncs/)
>>
>>
>> My understanding of the master/detail pattern we are using for our other
>> resources is that each task's URI will look something like this:
>> /api/v3/tasks/file/syncs/<uuid>/
>>
>> If the above is true, that means the client has to be aware of
>> '/api/v3/tasks/file/syncs/<uuid>/' resource. The user would need to know how
>> to interpret the response from each task type. There could be an infinite
>> number of these tasks with many different arguments. So writing client code
>> handling all those responses could become a nightmare. Right now the user
>> just needs to know how to interpret the response from '/api/v3/tasks/<uuid>'
>> for all tasks.
>>
>>>
>>>
>>>>
>>>> I don't see the value added by the proposed change.
>>>
>>>
>>> Value #1: We have to do something to address the problem that
>>> adding/removing content to a repository without plugin input is incorrect.
>>> This proposal is one possibility, but it isn't valid to compare the value
>>> against doing nothing. Instead, if you don't like this option, we need to
>>> compare it to other proposals for how to involve plugins in tasks.
>>>
>>
>> We don't have a problem. I agree that a user needs to know a whole lot of
>> information to correctly create a working repository version for a complex
>> content type such as a Docker manifest. Without all this information on
>> hand, the user could easily create a broken repository version. However, a
>> rich client could solve that problem. If the plugin writer wants to support
>> simpler clients, she can provide an additional URL for handling the
>> validation and additional logic for creating a repository version. We should
>> probably have a recommended convention for plugin authors to use when adding
>> additional URLs.
>>
>>>
>>> Value #2: Plugin writers no longer need to dispatch tasks. Plugin writers
>>> shouldn't need to understand the complexities of our tasking system.
>>> Pulpcore can handle task dispatching generally, plugin writers just set
>>> simple class attributes instead.
>>>
>>
>> Dispatching a task is 1 line of code. Plugin writers should be aware of
>> resource locking and how it works. They get to decide what resources are
>> appropriate to lock when performing asynchronous tasks.
>>
>>>
>>> Value #3: It removes "action" endpoints and replaces them with RESTful
>>> "creation of task" POSTs.
>>>
>>
>> +1 to this. However, not enough value by itself.
>>
>>>
>>> Value #4: It creates a convention for endpoints. Right now we have
>>>
>>>
>>> v3/importers/file/1234/sync/
>>> v3/publishers/python/1234/publish/
>>>
>>> There is no logical place for add/remove to go because add/remove will
>>> not involve an importer. It will be irritating to the users if we don't
>>> encourage some consistency between plugins for the simplest Pulp
>>> functionality.
>>>
>>>
>>> Value #5: Task history becomes much more useful. Currently, the only data
>>> on a task that connects it to user-facing objects is "created_resource".
>>> This proposal will allow users to filter tasks by parameters. For example,
>>> they can filter sync tasks by "repository" or "importer". They can also use
>>> the detail endpoints (v3/tasks/file/ or v3/tasks/file/syncs/) to list tasks
>>> based on the plugin or action.
>>>
>>
>> I don't think tracking history is one of the use cases we decided to
>> support in the MVP. We should have a separate discussion on how we could
>> accomplish this.
>>
>>>
>>> Value #6: Since the parameters of Task are saved, it will be possible to
>>> repeat tasks.
>>
>>
>> Same thing as for #5.
>>
>>
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>




More information about the Pulp-dev mailing list