[libvirt] Tunnelled post-copy live migration - support of bidirectional communication over the tunnel

[Daniel P. Berrange]

On Thu, Jan 15, 2015 at 02:45:13PM +0100, Vojtech Cima wrote:
> Hello everyone,
> 
> we are very interested about the post-copy live migration mechanism in
> libvirt [1] using QEMU. Mentioning post-copy in this context means the setup
> with the first full iteration of standard pre-copy and then switching to the
> post-copy. VIR_MIGRATE_ENABLE_POSTCOPY, VIR_MIGRATE_POSTCOPY_AFTER_PRECOPY
> together with VIR_MIGRATE_TUNNELLED flag.
> 
> Recently we have noticed the issue that post-copy live migration doesn't
> work over the tunnel since since current tunnel implementation in libvirt
> supports only uni-directional communication. I spent some time trying to
> enable bidirectional communication over the tunnel but still facing several
> issues and I would appreciate any hints how to move forward.
> 
> The idea is to add a function (probably a callback function) responsible for
> handling the traffic in the opposite direction (dest->src) reading from the
> virStream and writing to the QEMU [2]. To be able to do so I replaced the
> current 'pipe' calls with the 'socketpair' calls to create bidirectional
> paths.
> 
> Now, trying to use tunnelled post-copy live migration, it gets initiated,
> there is some traffic going from source to destination (obviously the
> pre-copy phase) but at some point it gets stuck on the 'saferead' function
> [3] which is now blocking. Can this issue issue be linked with the default
> socket blocking policy?
> Trying to use standard 'read' function instead of 'saferead' results that
> the lower data transfers and it seems it's missing significant part of
> information.

Since the code was historically unidirectional, the qemuMigrationIOFunc()
method used the virStream object in blocking I/O mode, and used a simple
loop to read from the pipe / write to the stream. If you want to do
bi-directional transfer that won't be possible - you'll need to put the
stream into fully non-blocking mode and use event callbacks to deal with
all I/O. This also means you can't use saferead either. The code in
tools/virsh-console.c is a reasonably good example of how to use a
stream in non-blocking mode doing  bi-directional I/O between the stream
and regular file descriptors. The virsh-console uses stdin/stdout but
you'd just use your socketpair instead.


> Another point is that we suppose that the virStream on its own supports
> bidirectional communication, is that correct?

Yes, absolutely.

> 
> [1] https://github.com/orbitfp7/qemu/tree/wp3-postcopy
> 
> [2] http://libvirt.org/git/?p=libvirt.git;a=blob;f=src/qemu/qemu_migration.c;h=0acbb5795e930e27157c580a3236701acd91bbaf;hb=e3435caf6af41748204e542dee13ede8441d88c0#l3595
> 
> [3] http://libvirt.org/git/?p=libvirt.git;a=blob;f=src/qemu/qemu_migration.c;h=0acbb5795e930e27157c580a3236701acd91bbaf;hb=e3435caf6af41748204e542dee13ede8441d88c0#l3644
> 
> Thank you very much for any ideas on this topic.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|