support for live migration with PCI passthrough devices

Laine Stump laine at redhat.com
Tue Aug 25 16:08:14 UTC 2020


On 8/25/20 8:56 AM, Michal Privoznik wrote:
> On 8/25/20 1:40 PM, Henry lol wrote:
>> Hi guys,
>>
>> I'm wondering whether libvirt supports live migration for the VM with 
>> PCI passthrough devices.
>> or it must be assumed before live migration that all passthrough 
>> devices be unplugged?
> 
> Unfortunately, this is still not supported. The problem is that PCI 
> devices themselves are not capable of dumping their internal state and 
> restoring from it on destination.
> 
> There is a long thread started last month that discuss what the 
> interface should look like, but at this point I guess we are still far 
> away from it:
> 
> https://www.redhat.com/archives/libvir-list/2020-July/msg00675.html
> 
>>
>> If so, all unplugged devices should be manually hot-plugged to the VM 
>> after migration??
> 
> This is the usual mode of operation, yes.

There is one exception to this - if the device to be migrated is the VF 
of an SRIOV-capable network card, then there is machinery in qemu to 
automatically unplug the device before migration starts, and then 
automatically plug in a new similar device on the other end after the 
guest has started there.

This all works in concert with a virtio-net emulated network device that 
is bonded together in the guest (linux only) to provide a single network 
device in the guest - when the VF is visible and working, it will be 
preferred for all traffic, but if the VF is missing then the virtio-net 
emulated device will be used.

The result is that under normal operation the vfio-assigned VF is used 
for all network traffic, but during migration when the VF has been 
unplugged, the virtio-net device is used, so there is no disruption in 
network traffic, just a bit of degraded performance during migration.

Here is a description of how this is configured in a libvirt domain 
definition:

 
https://www.libvirt.org/formatdomain.html#teaming-a-virtio-hostdev-nic-pair


This is all fairly new, and unfortunately doesn't work properly for all 
SRIOV cards.




More information about the libvirt-users mailing list