[libvirt] [RFC][scale] new API for querying domains stats

[Daniel P. Berrange]

On Thu, Jul 03, 2014 at 01:49:41PM -0600, Eric Blake wrote:
> On 07/01/2014 03:33 AM, Daniel P. Berrange wrote:
> 
> >  1. Time to write() the RPC call to the socket
> >  2. Time for libvirtd to process the RPC call
> >  3. Time to recv() the RPC reply from the socket
> >  ...and so on..
> > 
> > If the time for item 2 dominates over the time for items 1 & 2 (which
> > it should really) then the client thread is going to be sleeping in a
> > poll() for the bulk of the duration of the libvirt API call. If we had
> > an async API mechanism, then the VDSM time would essentially be consumed
> > with
> > 
> >  1. Time to write() the RPC call to the socket
> >  2. Time to write() the RPC call to the socket
> >  3. Time to write() the RPC call to the socket
> >  4. Time to write() the RPC call to the socket
> >  5. Time to write() the RPC call to the socket
> >  6. Time to write() the RPC call to the socket
> >  7. wait for replies to start arriving
> >  8. Time to recv() the RPC reply from the socket
> >  9. Time to recv() the RPC reply from the socket
> >  10. Time to recv() the RPC reply from the socket
> >  11. Time to recv() the RPC reply from the socket
> >  12. Time to recv() the RPC reply from the socket
> >  13. Time to recv() the RPC reply from the socket
> >  14. Time to recv() the RPC reply from the socket
> 
> This assumes you are still calling one async call per domain query.
> 
> With regards to a bulk API, are you thinking synchronous?
> 
> 1. Time to write() the RPC call - one bulk request
> 2. wait for reply - oh, and we'd better increase our on-wire size limits
> 3. Time to recv() the RPC reply - one bulk response
> 
> or asynchronous?
> 
> 1. Time to write() the RPC call - one bulk request
> 2. wait for replies to start arriving
> 3. Time to recv() an RPC async reply - first domain
> 4. Time to recv() an RPC async reply - second domain
> ...
> n. Time to recv() final RPC async reply
> 
> The asynchronous works nicely in that we don't have to size up our max
> RPC on-wire limits, but implies that you still need a callback invoked
> once per reply received, instead of getting all data back in one giant
> memory blob.

I was thinking the former actually, but the latter is another possibility
to consider I guess.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|