[libvirt] [RFC][scale] new API for querying domains stats
Richard W.M. Jones
rjones at redhat.com
Fri Jul 4 11:11:54 UTC 2014
On Tue, Jul 01, 2014 at 03:09:13AM -0400, Francesco Romani wrote:
> I'd like to discuss possible APIs and plans for new query APIs in libvirt.
> I'm one of the oVirt (http://www.ovirt.org) developers, and I write code for VDSM;
> VDSM is the node management daemon, which is in charge, among many other things, to
> gather the host and statistics per Domain/VM.
> Right now we aim for a number of VM per node in the (few) hundreds, but we have big plans
> to scale much more, and to possibly reach thousands in a not so distant future.
> At the moment, we use one thread per VM to gather the VM stats (CPU, network, disk),
> and of course this obviously scales poorly.
I'll just note here that a bug has been opened for virt-top, which
is similar to this.
If a domain has a large number of disks (256 virtio-scsi disks in the
customer's case), then virt-top spends so long fetching the data for
each separate disk, it can take 30-40 seconds between updates.
The same thing would happen if you had lots of domains, each with a
few disks, but with the total adding up to hundreds of disks.
The same thing would happen if you substitute network interfaces for disks.
What would help for us:
- A way to get information for multiple objects in a single domain
- A way to get information for multiple objects across multiple domains
in as few API round trips as possible.
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
More information about the libvir-list