[Pulp-dev] Task tagging in Pulp 3

[Michael Hrivnak]

On Mon, Nov 6, 2017 at 2:17 PM, David Davis <daviddavis at redhat.com> wrote:

> Originally I scheduled a meeting for tomorrow but on second thought, I
> figured a pulp-dev thread would be more inclusive than a meeting. I hope to
> get this resolved by the end of this week and if not then maybe we can have
> a meeting.
>
> This is to design out the replacement of task tags in Pulp 3. I’ve got
> feedback from a few other developers in terms of how to do that so I wrote
> up a sort of outline of the problem and two possible proposals. Looking for
> feedback/questions/etc on what people prefer.
>
>
> Background
> ---
>
> In Pulp 2, tasks have tags that either provide a task name/description or
> info on what resources a task acts on. Tasks also have reserved resources,
> which provide a way for tasks to lock a particular resource.
>

To provide a little more background, the tags are there 100% for clients.
It's a way for a client to find and display tasks based on a specific
resource, and maybe of a specific type (sync, publish, etc).

Reserved resources exist for business logic on the server side that is
mostly transparent to the client and user.


>
> In Pulp 3, we have models TaskTag and ReservedResource[0]. Tasks are
> associated with the resources they work on via TaskTag. If a resource is
> locked, a ReservedResource record is created in the db and then removed
> from the db once the resource is unlocked.
>
>
> Problem
> ---
>
> The task tag model doesn't really fit Pulp 3. It's perhaps too generic and
> totally unnecessary (see Proposal 1), or it could be redesigned to
> accomodate other things (see Proposal 2).
>
> Also, we need to support created resources (e.g. publications) with tasks.
> Refactoring task tags might provide an opportunity to do so.
>
>
> User stories
> ---
>
> As an authenticated user, I can see what resource(s) a task acted on.
> As an authenticated user, I can search for a tasks based on what resource
> they acted on.
>
>
> Proposal 1
> ---
>
> Since tags and reserved resources in Pulp 3 will only store information
> about a particular repository (not 100% sure here), it should be possible
> to simplify the data model. We could ditch both TaskTag and
> ReservedResource models and just have a direct relationship between Tasks
> and Repositories (e.g. TaskRepository). This model could also have some
> sort of field to indicate whether a particular field is locked (e.g.
> is_locked). Unlike ReservedResource, this relationship would be
> persisted—only the is_locked field would be updated when a task is done.
>

Using a repo and locking a repo are different relationships. Consider an
operation that only reads a certain repository and does not need to lock
it, such as the dep solving we were talking about today, or applicability
calculation. Or consider a publish operation on a repo version, which is
immutable; it would not need to lock the repo, but would still use it.


>
>
> Proposal 2
> ---
>
> We could keep the TaskTag relationship (perhaps even rename it to
> TaskResource) and we could add a field to indicate the nature of the
> relationship between task and resource (e.g. created, updated, etc). This
> field could not only capture what TaskTag is currently used for but also
> stuff like created resources (e.g. publications). We could also have a
> field to indicate which task resources are locked (e.g. is_locked).
>

That's very interesting. I like the idea of RESTful references to resources
that get used, and adding a qualifier for the type of relationship
(created, read, locked, deleted, etc) could be very useful.

If we used strong relationships instead of a string field to identify
reserved resources, one thing we would lose is the ability to lock on
something more abstract than a resource. We've done that in the past, but I
don't think anyone's loved it, so maybe we don't need it. As an example, if
we want only one orphan-purge task to run at a time, we can reserve a
resource named "orphan-removal". We did this for applicability calculation
for some time in Pulp 2 when there were safety questions about running them
concurrently.


>
> This would be useful for https://pulp.plan.io/issues/3033.
>
>
> Questions
> ---
>
> - What proposal do we want to adopt?
> - When do we need to address these changes?
> - Do we really need to allow users to search tasks by a resource/repo at
> all?
>

 I think they do need the ability to see what tasks are queued for a given
repo, and also what tasks have been recently completed.

-- 

Michael Hrivnak

Principal Software Engineer, RHCE

Red Hat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20171107/d54d33da/attachment.htm>