[Ovirt-devel] Re: Thoughts about taskomatic redesign

[David Lutterkort]

On Fri, 2008-06-27 at 10:10 -0700, Ian Main wrote:
> > We want to have  'n' x (number of logical CPU cores)   worker processes
> > for a value of 'n' yet to be determined - if they're mostly waiting
> > on I/O, then 'n' can be pretty large. We'll have to just try it out and
> > see what a good number is.
> 
> I'm pretty sure it's all IO bound stuff.  I think the queues will have
> to be implemented serially 

Not sure I understand .. isn't the whole point of this to process tasks
in parallel as much as possible ?

> so most of time you're waiting around for the node to complete it's
> business, so basing it on CPU cores is probably not really relevant.
> Probably expanding dynamically based on queue size would work better I
> would think.

The most critical resource for a worker thread is going to be the memory
it uses. So you definitely want an upper bound on the number of workers
that could ever be started, and that should probably be based off the
amount of memory available - but there's nothing wrong with making that
a configurable fixed value for now.

David