[libvirt] [RFC][scale] new API for querying domains stats

[Richard W.M. Jones]

On Fri, Jul 04, 2014 at 12:33:27PM +0100, Daniel P. Berrange wrote:
> On Fri, Jul 04, 2014 at 12:11:54PM +0100, Richard W.M. Jones wrote:
> > On Tue, Jul 01, 2014 at 03:09:13AM -0400, Francesco Romani wrote:
> > > I'd like to discuss possible APIs and plans for new query APIs in libvirt.
> > > 
> > > I'm one of the oVirt (http://www.ovirt.org) developers, and I write code for VDSM;
> > > VDSM is the node management daemon, which is in charge, among many other things, to
> > > gather the host and statistics per Domain/VM.
> > > 
> > > Right now we aim for a number of VM per node in the (few) hundreds, but we have big plans
> > > to scale much more, and to possibly reach thousands in a not so distant future.
> > > At the moment, we use one thread per VM to gather the VM stats (CPU, network, disk),
> > > and of course this obviously scales poorly.
> > 
> > I'll just note here that a bug has been opened for virt-top, which
> > is similar to this.
> > 
> > If a domain has a large number of disks (256 virtio-scsi disks in the
> > customer's case), then virt-top spends so long fetching the data for
> > each separate disk, it can take 30-40 seconds between updates.
> > 
> > The same thing would happen if you had lots of domains, each with a
> > few disks, but with the total adding up to hundreds of disks.
> > 
> > The same thing would happen if you substitute network interfaces for disks.
> > 
> > What would help for us:
> > 
> >  - A way to get information for multiple objects in a single domain
> > 
> >  - A way to get information for multiple objects across multiple domains
> 
> I'd say that we want something similar to the virDomainListAllDomains()
> API for stats. ie we shouldn't try to pass in the full list of domains
> or paths we want info for. We should just list all domains, optionally
> using flags to filter based on some characteristic, eg exclude inactive.
> Similarly always list stats for all disks.

FYI for virt-top we only care about stats of all active domains, and
we only care about all disks & all network interfaces for domains
(ie. never any subset).

We also collect CPU time and memory usage per domain.

Of course this only applies to virt-top, not to other clients.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v