[libvirt] [PATCH v1 00/11] Rework storage migration

[Daniel P. Berrange]

On Tue, Nov 27, 2012 at 07:49:54PM +0100, Michal Privoznik wrote:
> This patch set re-implements migration with storage for enough new qemu.
> Currently, you can migrate a domain to a host without need for shared storage.
> This is done by setting 'blk' or 'inc' attribute (representing
> VIR_MIGRATE_NON_SHARED_DISK and VIR_MIGRATE_NON_SHARED_INC flags respectively)
> of 'migrate' monitor command. However, the qemu implementation is
> buggy and applications are advised to switch to new impementation
> which, moreover, offers some nice features, like migrating only explicitly
> specified disks.
> 
> The new functionality is controlled via 'nbd-server-*' and 'drive-mirror'
> commands. The flow is meant to look like this:
> 
> 1) User invokes libvirt's migrate functionality.
> 
> 2) libvirt checks that no block jobs are active on the source.
> 
> 3) libvirt starts the destination QEMU and sets up the NBD server using the
> nbd-server-start and nbd-server-add commands.
> 
> 4) libvirt starts drive-mirror with a destination pointing to the remote NBD
> server, for example nbd:host:port:exportname=diskname (where diskname is the
> -drive id specified on the destination).

This is where I have a problem with automatically using the new impl. If
we consider that we need to allow multiple concurrent migrations, We are
introducing a requirement to open arbitrary number of ports in the firewall.
This data stream is also cleartext without any encryption, even using an
encrypted qcow2 disk won't help, since we're transferring the logical guest
side blocks, not the physical host side blocks.

What you have here is fine in the direct migration case, since we already
require arbitrary open ports and non-encrypted data stream.

If the user has requested TUNNELLED migration, we need follow up work done.
We need the NBD server to be able to accept a pre-opened file descriptor
to rather than listening on a TCP host/port, and then for libvirtd to
tunnel the data stream somehow.  Alternatively just use the old impl in
the TUNNELLED case.

> 5) once all mirroring jobs reach steady state, libvirt invokes the migrate
> command.
> 
> 6) once migration completed, libvirt invokes the nbd-server-stop command on the
> destination QEMU.
> 
> If we just skip the 2nd step and there is an active block-job, qemu will fail in
> step 4. No big deal.
> Since we try to NOT break migration and keep things compatible, this feature is
> enabled iff both sides support it. Since there's obvious need for some data
> transfer between src and dst, I've put it into qemuCookieMigration:
> 
> 
> 1)  src -> dest: (QEMU_MIGRATION_PHASE_BEGIN3 -> QEMU_MIGRATION_PHASE_PREPARE)
>     <nbd>
>        <disk src='/var/lib/libvirt/images/f17.img' size='17179869184'/>
>     </nbd>
> 
>     Hey destination, I know how to use this cool new feature. Moreover,
>     these are the paths I'll send you. Each file is X bytes big.
>     It's one of the prerequisite - the file on dst exists and has at least the
>     same size as on dst.
> 
> 2)  dst -> src: (QEMU_MIGRATION_PHASE_PREPARE -> QEMU_MIGRATION_PHASE_PERFORM3)
>     <nbd port='X'/>
> 
>     Okay, I (destination) support this feature as well. I've created all
>     files as you (src) told me to and you can start rolling data. I am listening
>     on port X.
> 
> 3)  src -> dst: (QEMU_MIGRATION_PHASE_PERFORM3 -> QEMU_MIGRATION_PHASE_FINISH3)
>     <nbd port='-1'/>
> 
>     Migration completed, destination, you may shut the NBD server down.
> 
> 
> If either src or dst doesn't support NBD, it is not used and whole process fall
> backs to old implementation.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|