[Libguestfs] parallelization

[Daniel P. Berrange]

On Tue, Feb 26, 2013 at 02:56:53PM +0000, Richard W.M. Jones wrote:
> On Tue, Feb 26, 2013 at 09:36:30AM -0500, Colin Walters wrote:
> > So Richard recently blogged about running many libguestfs mounts in
> > parallel.
> > 
> > I actually want parallelization in the other way; I use guestmount to
> > access a disk image, and my program is multithreaded.  It does
> > read/write and most especially calls to link() and fdatasync() from
> > threads.
> > 
> > However, this all ends up being serialized, because the guestfsd
> > protocol is synchronous, and the daemon side is single-threaded.
> > Related to this, the qemu instance is always booted with only one
> > vCPU.
> > 
> > Have there been any thoughts on fixing this?
> 
> Yes.  However it requires changes to the virtio-serial protocol or
> perhaps its entire replacement with VMware's vsock protocol (currently
> being discussed in upstream qemu).  Without this, it's pretty clunky
> to open more than one connection from the library to the appliance.

That's if you need to open multiple connections though. Sounds like
in Colin's case he just has one multi-threaded app, in which case
all apps could just share one connection & have the RPC messages
multi-plexed as we do with libvirt. Of course still needs the daemon
to be multithreaded to get any benefit from that.

> It would also need guestfsd to become multi-threaded.  That's not so
> much of a problem (although it's certainly a bit of work), but solving
> the protocol problem would have to be done first.


Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|