<div dir="ltr">Just to be clear, neither of these solutions were intended to solve these user stories. To me, the immediate need that prompted this discussion was to answer the question "how do we store created objects on tasks?" This is for task #3033[0]. My goal was to design just enough to answer that question. <div> </div><div>[0] <a href="https://pulp.plan.io/issues/3033" target="_blank">https://pulp.plan.io/issues/3033</a></div><div class="gmail_extra"> <div><div class="m_5300337663613357202gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div> </div><div>David </div></div></div></div></div></div></div></div> <div class="gmail_quote">On Tue, Nov 14, 2017 at 3:24 PM, Jeff Ortel <<a href="mailto:jortel@redhat.com" target="_blank">jortel@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 11/14/2017 01:24 PM, Brian Bouterse wrote: > Thanks for all the discussion! I agree there are improvements to be made here. > > I don't think either of these proposals solve all the problems without creating a few new ones. Rather than > saying +1 to either, I want to talk about the goals and use cases a bit more. Here is a list of 3 related use > cases that Pulp currently *can't* do, along with some commentary on the state of why. Once we decide on what > we think users need, figuring out how to make it should be straightforward. > > 1) As a user, I can know if a task is current locking a worker or not. Say they need to take a worker offline. > They have no way of knowing if that is safe to do at this moment without this info. Pulp internally knows this > information, but I don't believe this is visible to the user currently. This is useful info for debugging that > we regularly have to pull from the db. Regardless of either proposal, we still need to decide if this will be > included in the Task viewset. I'm +1 to adding this use case to the MVP explicitly. Do people feel this use > case should be added to the MVP? +1 > > 2) As a user, I can filter for tasks by the resource locked, e.g. repo 'foo', without forming a special search > string to search by. Currently the 'resource' field in the TaskTag model stores a string like > 'repository:foo'. Even if you know the name 'foo' you need to search via substring (inefficient and maybe > dangerous).You also can't search by other properties like the UUID, feed, etc because to Pulp it's just a > string 'repository:foo'. It doesn't know that is actually repository xxxxx-yyyy-yyyy-zzzzzzz with a name='foo'. I would like to see this use case expanded (into several cases) to include a description of why a user wants to do this. What are they trying to accomplish. Like: "As a user, I want to search for tasks pending for a repository because I'm trying to understand why my task isn't running yet." This applies to #3 as well. > > 3) As a user, I can filter for tasks by operation type (e.g. sync). Currently we have no way to do this. The > data model doesn't even have a field to capture this information. This feature seems simple from a high level, > but determining the specific taxonomy of those operation types it can get messy. We have 'sync' and 'publish', > those are pretty clear. What about 'update' like a publisher/importer/repo attribute update? How about 'add' > and 'remove' content? What if both add and remove happen in the same operation? Is that two tags or a new one? > If we're going to talk about this feature we need to call out the use cases more specifically. A series of use > cases like: "As a user, I can filter for tasks labeled with the 'sync' operation" could work. > > Another way to accomplish use case (3) is to record the actual task name as a string, e.g. > 'pulpcore.app.tasks.importer.sync'. This won't work well either because we DRY up our tasks, especially > update, so I think the simple taxonomy is still the way forward for that feature. <a href="https://pulp.plan.io/issues/3038" rel="noreferrer" target="_blank">https://pulp.plan.io/issues/3038</a> Agreed. Exposing and relying on an implementation detail like "pulpcore.app.tasks.importer.sync" would be bad. > > Each proposal affects these use cases, but neither of them totally enables all of them. Aside from solving > them, what do others thing about these use cases and the current state of Pulp3 w.r.t them? Thanks for all the > discussion. > > -Brian > > > On Thu, Nov 9, 2017 at 3:43 PM, Dennis Kliban <<a href="mailto:dkliban@redhat.com" target="_blank">dkliban@redhat.com</a> <mailto:<a href="mailto:dkliban@redhat.com" target="_blank">dkliban@redhat.com</a>>> wrote: <div><div class="m_5300337663613357202h5">> > On Mon, Nov 6, 2017 at 2:17 PM, David Davis <<a href="mailto:daviddavis@redhat.com" target="_blank">daviddavis@redhat.com</a> <mailto:<a href="mailto:daviddavis@redhat.com" target="_blank">daviddavis@redhat.com</a>>> wrote: > > Originally I scheduled a meeting for tomorrow but on second thought, I figured a pulp-dev thread would > be more inclusive than a meeting. I hope to get this resolved by the end of this week and if not then > maybe we can have a meeting. > > This is to design out the replacement of task tags in Pulp 3. I’ve got feedback from a few other > developers in terms of how to do that so I wrote up a sort of outline of the problem and two possible > proposals. Looking for feedback/questions/etc on what people prefer. > > > Background > --- > > In Pulp 2, tasks have tags that either provide a task name/description or info on what resources a > task acts on. Tasks also have reserved resources, which provide a way for tasks to lock a particular > resource. > > In Pulp 3, we have models TaskTag and ReservedResource[0]. Tasks are associated with the resources > they work on via TaskTag. If a resource is locked, a ReservedResource record is created in the db and > then removed from the db once the resource is unlocked. > > > Problem > --- > > The task tag model doesn't really fit Pulp 3. It's perhaps too generic and totally unnecessary (see > Proposal 1), or it could be redesigned to accomodate other things (see Proposal 2). > > Also, we need to support created resources (e.g. publications) with tasks. Refactoring task tags might > provide an opportunity to do so. > > > User stories > --- > > As an authenticated user, I can see what resource(s) a task acted on. > As an authenticated user, I can search for a tasks based on what resource they acted on. > > > Proposal 1 > --- > > Since tags and reserved resources in Pulp 3 will only store information about a particular repository > (not 100% sure here), it should be possible to simplify the data model. We could ditch both TaskTag > and ReservedResource models and just have a direct relationship between Tasks and Repositories (e.g. > TaskRepository). This model could also have some sort of field to indicate whether a particular field > is locked (e.g. is_locked). Unlike ReservedResource, this relationship would be persisted—only the > is_locked field would be updated when a task is done. > > > Proposal 2 > --- > > We could keep the TaskTag relationship (perhaps even rename it to TaskResource) and we could add a > field to indicate the nature of the relationship between task and resource (e.g. created, updated, > etc). This field could not only capture what TaskTag is currently used for but also stuff like created > resources (e.g. publications). We could also have a field to indicate which task resources are locked > (e.g. is_locked). > > > I like proposal number 2. > > > </div></div>> This would be useful for <a href="https://pulp.plan.io/issues/3033" rel="noreferrer" target="_blank">https://pulp.plan.io/issues/3033</a> <<a href="https://pulp.plan.io/issues/3033" rel="noreferrer" target="_blank">https://pulp.plan.io/issues/3033</a>>. > > > Questions > --- > > - What proposal do we want to adopt? > - When do we need to address these changes? > - Do we really need to allow users to search tasks by a resource/repo at all? > > > [0] <a href="https://git.io/vF8iH" rel="noreferrer" target="_blank">https://git.io/vF8iH</a> > > David > > _______________________________________________ > Pulp-dev mailing list > <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a>> > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> > > > > _______________________________________________ > Pulp-dev mailing list > <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> <mailto:<a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a>> > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> <<a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a>> <div class="m_5300337663613357202HOEnZb"><div class="m_5300337663613357202h5">> > > > > _______________________________________________ > Pulp-dev mailing list > <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> > <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> > </div></div> _______________________________________________ Pulp-dev mailing list <a href="mailto:Pulp-dev@redhat.com" target="_blank">Pulp-dev@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/pulp-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/pulp-dev</a> </blockquote></div> </div></div>