[Virtio-fs] Live migration support for virtio-fs

[Ge, Xiongzi]

Thanks, Stefan!
It seems that libvirt does not support device-attach for filesystem type device, like virtiofs. Are there any other ways to do this hot-unplugging? 

Xiongzi

On 4/8/21, 12:21 PM, "Stefan Hajnoczi" <stefanha at redhat.com> wrote:

    On Wed, Apr 07, 2021 at 06:05:29PM +0000, Ge, Xiongzi wrote:
    > Thanks Dr Gilbert, Vivek, Stefan, Greg!
    > I put together the discussion into this thread and CC qemu-devel at nongnu.org. 
    > 
    > Problem:
    > Current Virtio-FS does not support live migration.  Even when the virtiofs directory is not mounted on the guest, VM cannot do live migration. Any suggestions/interest If we want to make this work so we can still have the live migration feature for other purpose on that VM? 
    > 
    > If we already have a shared file system like Ceph, does this make it different? 

    (Marc-André: I CCed you regarding D-Bus VMState, see below.)

    During the last virtio-fs call on Wednesday we discussed how to allow
    live migration when the file system is not mounted. This means live
    migration of a mounted file system still wouldn't be supported, but at
    least it would be possible to migrate when the file system is not
    mounted.

    The easiest way to do that is to hot unplug the vhost-user-fs-pci device
    so that QEMU can live migrate the VM and then hot plug the
    vhost-user-fs-pci device again after migration has completed. This
    requires either manually performing the hot plug operations or some
    automation to do this around live migration.

    A smoother approach would be for QEMU to allow live migration when the
    file system is not mounted. Unfortunately it turns out this is actually
    rather involved because virtiofsd runs as a separate process from QEMU
    and the live migration interface for vhost-user devices is currently
    pretty basic
    (https://qemu.readthedocs.io/en/latest/interop/dbus-vmstate.html). There
    is currently no way for virtiofsd to communicate with QEMU to allow/deny
    live migration depending on whether virtiofsd currently has an active
    FUSE session (mount). It might be possible to extend the D-Bus VMState
    interface and implement support for "migration blockers" as they are
    known in QEMU, but I wonder whether it's not better to just work on
    full virtio-fs live migration instead. Either task is complex and I
    wouldn't recommend it as a first contribution to virtiofsd or QEMU.

    Can you make use of the hotplug approach to live migration?

    Stefan