[Pulp-dev] Task tagging in Pulp 3
David Davis
daviddavis at redhat.com
Tue Nov 14 21:41:46 UTC 2017
Just to be clear, neither of these solutions were intended to solve these
user stories. To me, the immediate need that prompted this discussion was
to answer the question "how do we store created objects on tasks?" This is
for task #3033[0]. My goal was to design just enough to answer that
question.
[0] https://pulp.plan.io/issues/3033
David
On Tue, Nov 14, 2017 at 3:24 PM, Jeff Ortel <jortel at redhat.com> wrote:
> On 11/14/2017 01:24 PM, Brian Bouterse wrote:
> > Thanks for all the discussion! I agree there are improvements to be made
> here.
> >
> > I don't think either of these proposals solve all the problems without
> creating a few new ones. Rather than
> > saying +1 to either, I want to talk about the goals and use cases a bit
> more. Here is a list of 3 related use
> > cases that Pulp currently *can't* do, along with some commentary on the
> state of why. Once we decide on what
> > we think users need, figuring out how to make it should be
> straightforward.
> >
> > 1) As a user, I can know if a task is current locking a worker or not.
> Say they need to take a worker offline.
> > They have no way of knowing if that is safe to do at this moment without
> this info. Pulp internally knows this
> > information, but I don't believe this is visible to the user currently.
> This is useful info for debugging that
> > we regularly have to pull from the db. Regardless of either proposal, we
> still need to decide if this will be
> > included in the Task viewset. I'm +1 to adding this use case to the MVP
> explicitly. Do people feel this use
> > case should be added to the MVP?
>
> +1
>
> >
> > 2) As a user, I can filter for tasks by the resource locked, e.g. repo
> 'foo', without forming a special search
> > string to search by. Currently the 'resource' field in the TaskTag model
> stores a string like
> > 'repository:foo'. Even if you know the name 'foo' you need to search via
> substring (inefficient and maybe
> > dangerous).You also can't search by other properties like the UUID,
> feed, etc because to Pulp it's just a
> > string 'repository:foo'. It doesn't know that is actually repository
> xxxxx-yyyy-yyyy-zzzzzzz with a name='foo'.
>
> I would like to see this use case expanded (into several cases) to include
> a description of why a user wants
> to do this. What are they trying to accomplish. Like: "As a user, I want
> to search for tasks pending for a
> repository because I'm trying to understand why my task isn't running yet."
>
> This applies to #3 as well.
>
> >
> > 3) As a user, I can filter for tasks by operation type (e.g. sync).
> Currently we have no way to do this. The
> > data model doesn't even have a field to capture this information. This
> feature seems simple from a high level,
> > but determining the specific taxonomy of those operation types it can
> get messy. We have 'sync' and 'publish',
> > those are pretty clear. What about 'update' like a
> publisher/importer/repo attribute update? How about 'add'
> > and 'remove' content? What if both add and remove happen in the same
> operation? Is that two tags or a new one?
> > If we're going to talk about this feature we need to call out the use
> cases more specifically. A series of use
> > cases like: "As a user, I can filter for tasks labeled with the 'sync'
> operation" could work.
> >
> > Another way to accomplish use case (3) is to record the actual task name
> as a string, e.g.
> > 'pulpcore.app.tasks.importer.sync'. This won't work well either because
> we DRY up our tasks, especially
> > update, so I think the simple taxonomy is still the way forward for that
> feature. https://pulp.plan.io/issues/3038
>
> Agreed. Exposing and relying on an implementation detail like
> "pulpcore.app.tasks.importer.sync" would be bad.
>
> >
> > Each proposal affects these use cases, but neither of them totally
> enables all of them. Aside from solving
> > them, what do others thing about these use cases and the current state
> of Pulp3 w.r.t them? Thanks for all the
> > discussion.
> >
> > -Brian
> >
> >
> > On Thu, Nov 9, 2017 at 3:43 PM, Dennis Kliban <dkliban at redhat.com
> <mailto:dkliban at redhat.com>> wrote:
> >
> > On Mon, Nov 6, 2017 at 2:17 PM, David Davis <daviddavis at redhat.com
> <mailto:daviddavis at redhat.com>> wrote:
> >
> > Originally I scheduled a meeting for tomorrow but on second
> thought, I figured a pulp-dev thread would
> > be more inclusive than a meeting. I hope to get this resolved by
> the end of this week and if not then
> > maybe we can have a meeting.
> >
> > This is to design out the replacement of task tags in Pulp 3.
> I’ve got feedback from a few other
> > developers in terms of how to do that so I wrote up a sort of
> outline of the problem and two possible
> > proposals. Looking for feedback/questions/etc on what people
> prefer.
> >
> >
> > Background
> > ---
> >
> > In Pulp 2, tasks have tags that either provide a task
> name/description or info on what resources a
> > task acts on. Tasks also have reserved resources, which provide
> a way for tasks to lock a particular
> > resource.
> >
> > In Pulp 3, we have models TaskTag and ReservedResource[0]. Tasks
> are associated with the resources
> > they work on via TaskTag. If a resource is locked, a
> ReservedResource record is created in the db and
> > then removed from the db once the resource is unlocked.
> >
> >
> > Problem
> > ---
> >
> > The task tag model doesn't really fit Pulp 3. It's perhaps too
> generic and totally unnecessary (see
> > Proposal 1), or it could be redesigned to accomodate other
> things (see Proposal 2).
> >
> > Also, we need to support created resources (e.g. publications)
> with tasks. Refactoring task tags might
> > provide an opportunity to do so.
> >
> >
> > User stories
> > ---
> >
> > As an authenticated user, I can see what resource(s) a task
> acted on.
> > As an authenticated user, I can search for a tasks based on what
> resource they acted on.
> >
> >
> > Proposal 1
> > ---
> >
> > Since tags and reserved resources in Pulp 3 will only store
> information about a particular repository
> > (not 100% sure here), it should be possible to simplify the data
> model. We could ditch both TaskTag
> > and ReservedResource models and just have a direct relationship
> between Tasks and Repositories (e.g.
> > TaskRepository). This model could also have some sort of field
> to indicate whether a particular field
> > is locked (e.g. is_locked). Unlike ReservedResource, this
> relationship would be persisted—only the
> > is_locked field would be updated when a task is done.
> >
> >
> > Proposal 2
> > ---
> >
> > We could keep the TaskTag relationship (perhaps even rename it
> to TaskResource) and we could add a
> > field to indicate the nature of the relationship between task
> and resource (e.g. created, updated,
> > etc). This field could not only capture what TaskTag is
> currently used for but also stuff like created
> > resources (e.g. publications). We could also have a field to
> indicate which task resources are locked
> > (e.g. is_locked).
> >
> >
> > I like proposal number 2.
> >
> >
> >
> > This would be useful for https://pulp.plan.io/issues/3033 <
> https://pulp.plan.io/issues/3033>.
> >
> >
> > Questions
> > ---
> >
> > - What proposal do we want to adopt?
> > - When do we need to address these changes?
> > - Do we really need to allow users to search tasks by a
> resource/repo at all?
> >
> >
> > [0] https://git.io/vF8iH
> >
> > David
> >
> > _______________________________________________
> > Pulp-dev mailing list
> > Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
> > https://www.redhat.com/mailman/listinfo/pulp-dev <
> https://www.redhat.com/mailman/listinfo/pulp-dev>
> >
> >
> >
> > _______________________________________________
> > Pulp-dev mailing list
> > Pulp-dev at redhat.com <mailto:Pulp-dev at redhat.com>
> > https://www.redhat.com/mailman/listinfo/pulp-dev <
> https://www.redhat.com/mailman/listinfo/pulp-dev>
> >
> >
> >
> >
> > _______________________________________________
> > Pulp-dev mailing list
> > Pulp-dev at redhat.com
> > https://www.redhat.com/mailman/listinfo/pulp-dev
> >
>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20171114/177e1de9/attachment.htm>
More information about the Pulp-dev
mailing list