[Ovirt-devel] Re: Thoughts about taskomatic redesign

[David Lutterkort]

On Fri, 2008-06-27 at 10:42 -0700, Ian Main wrote:
> On Wed, 25 Jun 2008 18:32:09 -0700
> Ian Main <imain at redhat.com> wrote:
> 
> OK, here is an update based on feedback and some more thinking.
> 
> Requirements again.. added one new one:
> 
> - NEW: Ability to accept tasks from WUI or scripting API. 
> - Execute many tasks in a timely manner (presumably via distributed/multi threaded setup)
> - Execute tasks in the correct order.
> - Implement transaction support (if we can/want it)
> - Have good error reporting.
> - Include authentication and encryption.
> - It would be nice to be able to see the state of the queues in the WUI.
> - Implement at least the following tasks: 
>    - start/stop/create/destroy VMs
>    - clear host functionality.  Destroy all VMs/pools.
>    - Migrate vms.
>    - Storage pool creation/destruction/management
> 
> If you start to think about the scripting API requirement you then
> have to consider how high level you want this API, and also about
> where to do task ordering and breakdown of tasks into actions if
> desired.  Any comments here?

I think exposing the API that adds individual tasks as the scripting API
would be the best option, so that that API supports the tasks you listed
above; the scripting API will also need to support some querying
functionality (like what pools exist etc.)

> It's pretty clear that we don't want to put the logic into the WUI as
> then it would need to be duplicated in a scripting API.

What are your thoughts on transport for the scripting API ? Some sort of
http/https based API seems the cleanest to me (like something RESTful)
In that case, it might make sense to run the WUI and the scripting API
side-by-side, since you're already running a webserver for the WUI.

> So I think:
> 
> - Task queuing API.  Provides logic for queuing tasks to the database
> and inserting dependency information.  We still have race conditions
> to contend with in terms of inserting new rows with dependency
> information.. would we need to lock the table to prevent this?

I think that's a question best answered once you have some code; how
exactly to guard inserts depends on the actual schema and queries used.

> I might be tempted to make the UI in qpid modeling and have a single
> process deal with this; this also allows all supported qpid modeling
> languages to be used.  

I don't think qpid enters into the task queue API ... it's only the
transport that is used to communicate between individual actions for
each task and the managed nodes. 

> At this point I'm thinking the API should be high level just like the
> WUI uses now.

Agreed.

> - Queuing can be done in postgresql DB as a single table.  We can look
> at using the NOTIFY optimization at some point if we like.  Each task
> should be placed in the database with a dependency ID of the previous
> task if required along with status fields etc.
> 
> - Multiple taskomatic threads/processes can run on the WUI or any
> other machine and query the DB for tasks.  It will select tasks which
> have no uncompleted dependencies, lock the row, set in-progress
> status, work on task, set completed/error conditions, unlock the row.

Sounds good.

> There seems to be no debate on using qpid/amqp modeling for
> communicating with the nodes so I guess we'll run with that.  I'm
> thinking thread count can be expanded whenever there are available
> tasks that aren't being dealt with.  Note that ruby doesn't actually
> implement real threads.

Since the threads will mostly be idle, I don't think that that is a huge
issue; I would implement whatever seems easiest and then see how it
falls apart ;)

David