Xen with libvirt and SR-IOV

[nospam at godawa.de]

Hi everybody,

since long time, I'm using Xen on CentOS with XL, currently latest 
CentOS 7 with Xen 4.15 from the CentOS-Xen-Project. For several VMs I 
have to use SR-IOV, to lower the CPU-usage on Dom0 on the host.

CentOS 7 comes to an end, Xen is not supported by RHEL nor RockyLinux 
anymore, so unfortunately I have to switch to KVM.

First step will be now, converting all the scripts for managing and 
running VMs, that they run with the additional libvirt-layer.

Mostly everything is working, but I do not get a network interface in 
the VM, when I start it with "virsh start ..." instead of "xl create ...".


First of all, is there documentation how to configure the VMs in the 
dom-definition for XEN (all docs I found are KVM-related)?

The converted xl-config does not do the job:

virsh -c xen:/// domxml-from-native --format xen-xl vm > vm.xml

These are some none working examples I tried out:

     ...
     <interface type='hostdev' managed='yes'>
       <mac address='02:16:32:10:20:30'/>
       <driver name='xen'/>
       <source>
         <address type='pci' domain='0x0000' bus='0x81' slot='0x02' 
function='0x6'/>
       </source>
       <vlan>
         <tag id='11'/>
       </vlan>
     </interface>
     ...


     ...
     <hostdev mode='subsystem' type='pci' managed='yes'>
       <driver name='xen'/>
       <source>
         <address domain='0x0000' bus='0x81' slot='0x02' function='0x6'/>
       </source>
       <address type='pci' domain='0x0000' bus='0x81' slot='0x02' 
function='0x6'/>
     </hostdev>
     ...


Result is always the same, the VM does not find any interface to 
configure, when starting as libvirt:

# dmesg | egrep -i "net|eth"
...
[    4.523173] iavf: Intel(R) Ethernet Adaptive Virtual Function Network 
Driver - version 4.4.2.1

# lspci | egrep -i "net|eth"
00:00.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 
700 Series (rev 02)

# lspci -vmmks 00:00.6
Slot:	00:00.6
Class:	Ethernet controller
Vendor:	Intel Corporation
Device:	Ethernet Virtual Function 700 Series
SVendor:	Intel Corporation
SDevice:	Device 0000
Rev:	02
Module:	i40evf
Module:	iavf
NUMANode:	0

# lsmod | egrep -i "iavf|i40"
iavf                  135168  0
auxiliary              16384  1 iavf
ptp                    20480  1 iavf

# ifconfig eth0
eth0: error fetching interface information: Device not found


The same VM after starting with XL:

# dmesg | egrep -i "net|eth"
...
[    4.742038] iavf: Intel(R) Ethernet Adaptive Virtual Function Network 
Driver - version 4.4.2.1
[   40.578461] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   40.642868] iavf 0000:00:00.6 eth0: NIC Link is Up Speed is 10 Gbps 
Full Duplex
[   40.644015] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

# lspci | egrep -i "net|eth"
00:00.6 Ethernet controller: Intel Corporation Ethernet Virtual Function 
700 Series (rev 02)

# lspci -vmmks 00:00.6
Slot:	00:00.6
Class:	Ethernet controller
Vendor:	Intel Corporation
Device:	Ethernet Virtual Function 700 Series
SVendor:	Intel Corporation
SDevice:	Device 0000
Rev:	02
Driver:	iavf
Module:	i40evf
Module:	iavf
NUMANode:	0

# lsmod | egrep -i "iavf|i40"
iavf                  135168  0
auxiliary              16384  1 iavf
ptp                    20480  1 iavf

# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
         inet 10.20.30.40  netmask 255.255.255.0  broadcast 10.20.30.1
         ...
         ether 02:16:32:10:20:30  txqueuelen 1000  (Ethernet)


I expect, that all the SR-IOV stuff is configured correctly, because the 
VMs runs in XL-mode without any problems.


Thanks a lot for any ideas,
-- 

kind regards,

Thorolf