[libvirt] [RFC PATCH 2/2] lib: Add note that bulk stats API queries may overrun RPC buffers

[Peter Krempa]

On Tue, May 23, 2017 at 10:04:44 +0100, Daniel Berrange wrote:
> On Tue, May 23, 2017 at 10:51:47AM +0200, Martin Kletzander wrote:
> > On Tue, May 23, 2017 at 09:32:03AM +0100, Daniel P. Berrange wrote:
> > > On Mon, May 22, 2017 at 06:00:13PM +0200, Peter Krempa wrote:
> > > > Hint that the users should limit the number of VMs queried in the bulk
> > > > stats API.

[...]

> > Or we can give a hint on what the limit is and let users figure out the
> > sensible number.  It is not only based on the number of VMs.  I believe
> > you can hit the limit with one or two VMs if you have lot of devices
> > that we report statistics for.  Limiting the number of VMs to a
> > particular number would not help as much in this case.  But we can
> > combine both approaches.
> 
> Urgh, that's even worse. Apps can't simply split queries into blocks of
> N vms, because any single VM in that list might have huge number of disks.
> So to use this at all reliably you have to query the XML config of every
> guest to see what devices are present and then split up the queries into
> variable number of VMs :-(
> 
> This just adds to my feeling that we should consider this API a failed
> experiment

Given this metric, any API in libvirt that takes XML is failed in the
same sense. You can easily have a VM that exceeds the 4MiB string RPC
limit (and 16, or 32) MiB in this sense anyways.

Along with metadata, you can reach the limit even easier.

A disk information dump returned with this API has slightly above 600
bytes, so let's say 1k, with this metric you can have a VM with 16k
disks/nics/whatever, which is reasonable. With the bump to 32MiB, you
can obviously have even more.

Since this API makes sense (saves time) even if called for a singe VM I
don't really think this is a big problem. If you say that the number of
VMs you should query is let's say 2 or 4, you can have giant guests.

The only non-scalable part is if you have lots of giant guests. You
can't really fix that.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20170523/e2ed40e7/attachment-0001.sig>