[libvirt] [RFC][scale] new API for querying domains stats

Richard W.M. Jones rjones at redhat.com
Fri Jul 4 11:11:54 UTC 2014


On Tue, Jul 01, 2014 at 03:09:13AM -0400, Francesco Romani wrote:
> I'd like to discuss possible APIs and plans for new query APIs in libvirt.
> 
> I'm one of the oVirt (http://www.ovirt.org) developers, and I write code for VDSM;
> VDSM is the node management daemon, which is in charge, among many other things, to
> gather the host and statistics per Domain/VM.
> 
> Right now we aim for a number of VM per node in the (few) hundreds, but we have big plans
> to scale much more, and to possibly reach thousands in a not so distant future.
> At the moment, we use one thread per VM to gather the VM stats (CPU, network, disk),
> and of course this obviously scales poorly.

I'll just note here that a bug has been opened for virt-top, which
is similar to this.

If a domain has a large number of disks (256 virtio-scsi disks in the
customer's case), then virt-top spends so long fetching the data for
each separate disk, it can take 30-40 seconds between updates.

The same thing would happen if you had lots of domains, each with a
few disks, but with the total adding up to hundreds of disks.

The same thing would happen if you substitute network interfaces for disks.

What would help for us:

 - A way to get information for multiple objects in a single domain

 - A way to get information for multiple objects across multiple domains

in as few API round trips as possible.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/




More information about the libvir-list mailing list