[Ovirt-devel] Re: Thoughts about taskomatic redesign

[Scott Seago]

David Lutterkort wrote:
> On Fri, 2008-06-27 at 10:24 -0400, Scott Seago wrote:
>   
>> David Lutterkort wrote:
>>     
>>> On Wed, 2008-06-25 at 18:32 -0700, Ian Main wrote:
>>>
>>>       
>> What we _don't_ have in the current implementation is:
>>   1) any notion of task dependencies (i.e. task 66 must run after 
>> 33,44,and 55 have completed. 33 and 44 must complete successfully, but 
>> success state of 55 is unimportant)
>>     
>
> Are optional dependencies really needed ? I would just stick with hard
> dependencies if at all possible.
>
>   
I'm not certain they're needed, so we can leave it out until we have a 
use case. I'm not sure 'optional' is the right term here, though. My 
thinking was there may be some sort of dependency relationship where you 
have two tasks where you need to preserver order of operation but don't 
necessarily require the first to succeed before running the second. Once 
we start defining the dependency relationships it will be clearer 
whether or not we need this bit.
>>   2) any notion of compound tasks (i.e. task 100 is composed of subtasks 
>> 101,102,and 103. These subtasks have their own dependencies and are 
>> tracked just as top-level tasks, and task 100 is essentialy a no-op task 
>> with dependencies on its subtasks
>>     
>
> Probably another thing that's best left as 'future work'. From your
> description of existing tasks, it seems that keeping them close to the
> WUI interaction is probably the simplest for now.
>
>   
That's fine. Without this, though, it might be harder to define 
dependencies, since the 'clear this host' tasks will have a much more 
complex dependency graph. i.e. any prior or subsequent queued tasks on 
any of the VMs on this host will block on this. If we have subtasks, 
then if a host has 15 VMs on it, later vm tasks for those VMs will only 
block until each individual VM has migrated rather than waiting for the 
whole lot of them to complete.
>> We may not need two-level queuing, but generating the dependency graph 
>> will have to happen at one step -- "order the tasks" as stated above 
>> will still be necessary, but it's more a case of "work out the 
>> dependencies for each task".
>>     
>
> Unless it's needed for some spiffy display in the WUI, there's really no
> reason to ever handle the dependency graph as a whole. All you need are
> ways to pick out 'runnable' tasks and/or find existing tasks that a new
> task may depend on.
>
>   
True. I wasn't thinking of acting on the whole graph -- rather for each 
new task identifying which current 'new' tasks are dependencies and 
insert the appropriate rows to track this. That way 'runnable' tasks is 
a simple query. a task is runnable if all of the dependencies are done.
>>  When the WUI insterts a task for "create 
>> this VM" or "migrate this VM" the dependency information will not be 
>> known by the WUI. So something on the taskomatic side will have to, as 
>> part of the "notice new tasks in the queue and process them"
>>     
>
> That's an important point and reason to separate the task queueing logic
> a little more from the WUI, so that instead of just creating and saving
> a new task object to the DB, you actually have some sort of API for
> adding new tasks, and the implementation of the API knows what possible
> dependencies and prerequisites to look for for each task.
>
>   
True. There are two steps needed here. "insert this task" and "figure 
out what it depends on". There are various ways to model this:
1) queueing API figures out current deps based on what's in the queue 
before (or perhaps just after) insertion
2) queuing API just inserts the task and whatever API looks for new 
tasks does the dependency checking.

It actually probably makes more sense for 1) to happen. Although this 
API could just be built into the Task ActiveRecord model -- then the WUI 
would just call Task.insert rather than Task.new.

> That API would probably also be the right one to expose publically.
>
>   
Right.  Perhaps this public API would be a layer between the current 
ruby model and controller classes? (then exposed via REST or whatever) 
-- or would the APIs be built into the model classes themselves (again 
exposed in the same way)?
> David
>
>
>   
Scott