[libvirt] [RFC v1 4/6] migration: Migration support for ephemeral hostdevs

[Daniel P. Berrange]

On Wed, May 13, 2015 at 11:36:30AM +0800, Chen Fan wrote:
> add migration support for ephemeral host devices, introduce
> two 'detach' and 'restore' functions to unplug/plug host devices
> during migration.
> 
> Signed-off-by: Chen Fan <chen.fan.fnst at cn.fujitsu.com>
> ---
>  src/qemu/qemu_migration.c | 171 ++++++++++++++++++++++++++++++++++++++++++++--
>  src/qemu/qemu_migration.h |   9 +++
>  src/qemu/qemu_process.c   |  11 +++
>  3 files changed, 187 insertions(+), 4 deletions(-)
> 
> diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c
> index 56112f9..d5a698f 100644
> --- a/src/qemu/qemu_migration.c
> +++ b/src/qemu/qemu_migration.c

> +void
> +qemuMigrationRestoreEphemeralDevices(virQEMUDriverPtr driver,
> +                                     virConnectPtr conn,
> +                                     virDomainObjPtr vm,
> +                                     bool live)
> +{
> +    qemuDomainObjPrivatePtr priv = vm->privateData;
> +    virDomainDeviceDefPtr dev;
> +    int ret = -1;
> +    size_t i;
> +
> +    VIR_DEBUG("Rum domain restore ephemeral devices");
> +
> +    for (i = 0; i < priv->nEphemeralDevices; i++) {
> +        dev = priv->ephemeralDevices[i];
> +
> +        switch ((virDomainDeviceType) dev->type) {
> +        case VIR_DOMAIN_DEVICE_NET:
> +            if (live) {
> +                ret = qemuDomainAttachNetDevice(conn, driver, vm,
> +                                                dev->data.net);
> +            } else {
> +                ret = virDomainNetInsert(vm->def, dev->data.net);
> +            }
> +
> +            if (!ret)
> +                dev->data.net = NULL;
> +            break;
> +        case VIR_DOMAIN_DEVICE_HOSTDEV:
> +            if (live) {
> +                ret = qemuDomainAttachHostDevice(conn, driver, vm,
> +                                                 dev->data.hostdev);
> +           } else {
> +                ret =virDomainHostdevInsert(vm->def, dev->data.hostdev);
> +            }

This re-attach step is where we actually have far far far worse problems
than with detach. This is blindly assuming that the guest on the target
host can use the same hostdev that it was using on the source host. This
is essentially useless in the real world. Even if the same vendor/model
device is available on the target host, it is very unlikely to be available
at the same bus/slot/function that it was on the source. It is quite likely
neccessary to allocate a complete different NIC, or if using SRIOV allocate
a different function. It is also not uncommon to have different vendor/models,
so a completely different NIC may be required.

It is impossible for libvirt todo anything sensible when picking the hostdev
to use on the target host as it does not have anywhere near enough knowledge
to make a correct decision. For example, it does not know which physical
network each NIC on the target host is plugged into. Even if it knew the
networks, it does not know what the I/O utilization is likel, to be able
to intelligently decide between a set of possible free NICs. In any non-trivial
mgmt app, the management app itself will have this knowledge and have policies
around which hostdevice to assign to a guest given a particular set of
circumstances. It may even decide not to assign a hostdev on the target and
instead provide 2 or 3 or more emulated devices that could be used in
bandwidth aggregation mode rather than failover mode.

In OpenStack, the compute hosts don't even decide which NICs are given to
which guests. This is down to an external schedular running on a different
host(s), and the compute host just hotplugs what has already been decided
elsewhere.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|