[Ovirt-devel] Re: Thoughts about taskomatic redesign

[Ian Main]

On Fri, 27 Jun 2008 18:04:31 +0000
David Lutterkort <dlutter at redhat.com> wrote:

> On Fri, 2008-06-27 at 10:10 -0700, Ian Main wrote:
> > > We want to have  'n' x (number of logical CPU cores)   worker processes
> > > for a value of 'n' yet to be determined - if they're mostly waiting
> > > on I/O, then 'n' can be pretty large. We'll have to just try it out and
> > > see what a good number is.
> > 
> > I'm pretty sure it's all IO bound stuff.  I think the queues will have
> > to be implemented serially 
> 
> Not sure I understand .. isn't the whole point of this to process tasks
> in parallel as much as possible ?

Huh, must have had a brain fart or something.  Ah, I know what I meant.. I meant non-async calls to the nodes to implement the tasks.  So they'll be spending a lot of time waiting for the return.

> > so most of time you're waiting around for the node to complete it's
> > business, so basing it on CPU cores is probably not really relevant.
> > Probably expanding dynamically based on queue size would work better I
> > would think.
> 
> The most critical resource for a worker thread is going to be the memory
> it uses. So you definitely want an upper bound on the number of workers
> that could ever be started, and that should probably be based off the
> amount of memory available - but there's nothing wrong with making that
> a configurable fixed value for now.

Good point.

	Ian