[Libvir] Proposal: Block device and network stats

[Daniel P. Berrange]

On Fri, Aug 10, 2007 at 02:58:59PM +0100, Richard W.M. Jones wrote:
> Daniel Veillard wrote:
> >   wr_bytes :-)
> 
> Duh :-(
> 
> >>  int64_t errs;   // In Xen this returns the mysterious 'oo_req'.
> >>};
> >>typedef struct _virDomainBlockStats *virDomainBlockStatsPtr;
> >
> >  then yes that's okay.
> >Those interfaces are what I suggested as the low level ones, I guess
> >they are needed even if tehy don't really scale as the number of 
> >domain and more importantly the number of node increases. But it allows
> >to build higher level monitoring implementations.
> >My current POV based on previous monitoring work is that if you are 
> >monitoring
> >up to a few dozen machines then aggregating the data to the monitoring 
> >application is fine, where you can then apply the user based policy to
> >raise the events (and subsequent rules or UI alerts), but if you want
> >to scale you have to export the monitoring down to each node, and push
> >the policies there, just gathering the events/alerts at the monitoring
> >application or console level.
> >In any case having the low level API is needed so something like those
> >entry points are needed.
> 
> Agreed.
> 
> The other issue is remote, where we make 2 * nr_domains round trips. 
> We're already making some k * nr_domains round trips to get the other 
> info (eg. domain names, state, VCPU usage, ...) so there is an argument 
> for being able to aggregate remote requests.  The easiest thing would 
> seem to be to allow remote to pipeline requests and responses. 
> Pipelining would get rid of the round trip delays.  The remote protocol 
> allows pipelining already, but needs some mucky coding on top to make it 
> actually work.

That would still require that we introduce a whole new set of APIs in 
the libvirt public interface, since all the current APIs are synchronous
so the caller waits for the reply - there'd never be anything to pipeline.

I think I'd like to see more APIs for geting info about domains in bulk,
because these would let us optimize the actual data retrieval in the drivers
themselves.

As an example - currently we have  virListDomainIDs, gives us an list of
IDs, and we then do virLookupDomainByID for each one to get a virDomainPtr.
Likewise virListDefinedDomains gives us a list of names and we then do
a virLookupDomainByName.  This requires O(n+m) calls to XenD.

If we had a virListDomains()  returning virDomainPtr objects directly for
both active & inactive domains then we could impl that in 1 call to XenD.

For monitoring while having APIs for getting stats about a single device
would be useful in some cases, if you're monitoring all domains I think
it would be worth having an API to get stats about all devices in all
domains at once. If it could also return all the virDomainInfo data at
once that'd be even more useful - you can fetch stats from the HV for all
guests in a single hypercall too

Regards,
Dan.
-- 
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=|