[libvirt] [Qemu-devel] [RFC 0/2] Attempt to implement the standby feature for assigned network devices

[Michael Roth]

Quoting Sameeh Jubran (2018-10-25 13:01:10)
> On Thu, Oct 25, 2018 at 5:06 PM Sameeh Jubran <sameeh at daynix.com> wrote:
> >
> > From: Sameeh Jubran <sjubran at redhat.com>
> >
> > Hi all,
> >
> > Background:
> >
> > There has been a few attempts to implement the standby feature for vfio
> > assigned devices which aims to enable the migration of such devices. This
> > is another attempt.
> >
> > The series implements an infrastructure for hiding devices from the bus
> > upon boot. What it does is the following:
> >
> > * In the first patch the infrastructure for hiding the device is added
> >   for the qbus and qdev APIs. A "hidden" boolean is added to the device
> >   state and it is set based on a callback to the standby device which
> >   registers itself for handling the assessment: "should the primary device
> >   be hidden?" by cross validating the ids of the devices.
> >
> > * In the second patch the virtio-net uses the API to hide the vfio
> >   device and unhides it when the feature is acked.
> >
> > Disclaimers:
> >
> > * I have only scratch tested this and from qemu side, it seems to be
> >   working.
> > * This is an RFC so it lacks some proper error handling in few cases
> >   and proper resource freeing. I wanted to get some feedback first
> >   before it is finalized.
> >
> > Command line example:
> >
> > /home/sameeh/Builds/failover/qemu/x86_64-softmmu/qemu-system-x86_64 \
> > -netdev tap,id=hostnet0,script=world_bridge_standalone.sh,downscript=no,ifname=cc1_71 \
> > -netdev tap,vhost=on,id=hostnet1,script=world_bridge_standalone.sh,downscript=no,ifname=cc1_72,queues=4 \
> > -device virtio-net,host_mtu=1500,netdev=hostnet1,id=cc1_72,vectors=10,mq=on,primary=cc1_71 \
> > -device e1000,netdev=hostnet0,id=cc1_71,standby=cc1_72 \
> >
> > Migration support:
> >
> > Pre migration or during setup phase of the migration we should send an
> > unplug request to the guest to unplug the primary device. I haven't had
> > the chance to implement that part yet but should do soon. Do you know
> > what's the best approach to do so? I wanted to have a callback to the
> > virtio-net device which tries to send an unplug request to the guest and
> > if succeeds then the migration continues. It needs to handle the case where
> > the migration fails and then it has to replug the primary device back.
> I think that the "add_migration_state_change_notifier" API call can be used
> from within the virtio-net device to achieve this, what do you think?

I think it would be good to hear from the libvirt folks (on Cc:) on this as
having QEMU unplug a device without libvirt's involvement seems like it
could cause issues. Personally I think it seems cleaner to just have QEMU
handle the 'hidden' aspects of the device and leave it to QMP/libvirt to do
the unplug beforehand. On the libvirt side I could imagine adding an option
like virsh migrate --switch-to-standby-networking or something along
that line to do it automatically (if we decide doing it automatically is
even needed on that end).

> >
> > The following terms are used as interchangeable:
> > standby - virtio-net
> > primary - vfio-device - physical device - assigned device
> >
> > Please share your thoughts and suggestions,
> > Thanks!
> >
> > Sameeh Jubran (2):
> >   qdev/qbus: Add hidden device support
> >   virtio-net: Implement VIRTIO_NET_F_STANDBY feature
> >
> >  hw/core/qdev.c                 | 48 +++++++++++++++++++++++++---
> >  hw/net/virtio-net.c            | 25 +++++++++++++++
> >  hw/pci/pci.c                   |  1 +
> >  include/hw/pci/pci.h           |  2 ++
> >  include/hw/qdev-core.h         | 11 ++++++-
> >  include/hw/virtio/virtio-net.h |  5 +++
> >  qdev-monitor.c                 | 58 ++++++++++++++++++++++++++++++++--
> >  7 files changed, 142 insertions(+), 8 deletions(-)
> >
> > --
> > 2.17.0
> >
> 
> 
> -- 
> Respectfully,
> Sameeh Jubran
> Linkedin
> Software Engineer @ Daynix.
>