[libvirt] Raising limits for our RPC messages

[Daniel P. Berrange]

On Thu, May 04, 2017 at 02:55:26PM +0300, Vasiliy Tolstov wrote:
> 2017-05-04 14:49 GMT+03:00 Daniel P. Berrange <berrange at redhat.com>:
> >
> > So I think it is reasonable to simply increase the buffer size as
> > we have done before.
> >
> > That said, we should bear this problem in mind before adding more
> > "bulk query" APIs, as it isn't sensible to carry on increasing
> > RPC message size forever. As somepoint we have to consider that
> > serializing 100's of MB of data into an RPC message is an
> > inherantly inefficient design, and consider alternative API
> > designs without this.
> >
> > For bulk stats query we could do something totally radical and
> > have the facility  for the client to send us a shared memory
> > region which we asynchronously populate with stats, avoiding the
> > RPC layer entirely. Obviously only works for local connections,
> > but I get the impression that most mgmt apps have a node-local
> > agent talking to libvirtd anyway.
> 
> 
> Does it possible to compress/uncompress data before/after transmission?
> I think that stats data can be efficient compressed...

That doesn't really help memory usage, as you still have todo the XDR
encode step to create the data that you then feed to the compressor.
It only saves on amount of data going over the wire, at the cost of
burning CPU time on compression.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|