[libvirt] Error starting domain: internal error: missing IFLA_VF_INFO in netlink response

[Hong-Hua.Yin at freescale.com]

Hi Laine,

Thank you for the reply. I tried to dump the information.

> -----Original Message-----
> From: sendmail [mailto:justsendmailnothingelse at gmail.com] On Behalf Of Laine
> Stump
> Sent: Thursday, January 08, 2015 11:13 PM
> To: Yin Olivia-R63875
> Cc: libvir-list at redhat.com
> Subject: Re: Error starting domain: internal error: missing IFLA_VF_INFO in netlink
> response
> 
> On 01/08/2015 04:40 AM, Hong-Hua.Yin at freescale.com wrote:
> > Hi Laine,
> >
> > Sorry to disturb you.
> > It seemed this issue had been fixed in libvirt-1.2.2/libnl-3.2.22/linux-3.12. But
> we still got the error on PowerPC platform.
> > I'll appreciate if you could give any suggestion. We are not sure if any netlink
> implementation in kernel space is missed.
> >
> > The scenario is a little complicated. We installed internal PF and VF
> > kernel modules and want to use <interface type="hostdev" managed="yes">
> syntax to start a guest domain with MAC address.
> >
> > # insmod fslinic.ko max_vfs=2
> > Freescale 10 Gigabit PCI Express Network Driver fslinic 0000:01:00.0:
> > Multiqueue Enabled: Rx Queue count = 1, Tx Queue count = 1 fslinic
> > 0000:01:00.0: Freescale (R) 10 Gigabit Network Connection fslinic
> > 0000:01:00.1: Multiqueue Enabled: Rx Queue count = 1, Tx Queue count =
> > 1 fslinic 0000:01:00.1: Freescale (R) 10 Gigabit Network Connection
> >
> > # insmod fslinicvf.ko
> > Freescale 10 Gigabit PCI Express Network Driver
> >
> > # lspci -mk
> > 00:00.0 "PCI bridge" "Freescale Semiconductor Inc" "Device 0440" -r20 "" ""
> > 01:00.0 "Power PC" "Freescale Semiconductor Inc" "Device 0440" -r20 "" ""
> > 01:00.1 "Power PC" "Freescale Semiconductor Inc" "Device 0440" -r20 "" ""
> > 01:00.4 "Power PC" "Freescale Semiconductor Inc" "Device 0000" -r20 "" ""
> > 01:00.5 "Power PC" "Freescale Semiconductor Inc" "Device 1957" -r20 "" ""
> > 01:01.0 "Power PC" "Freescale Semiconductor Inc" "Device 0000" -r20 "" ""
> > 01:01.1 "Power PC" "Freescale Semiconductor Inc" "Device 1957" -r20 "" ""
> > 0001:00:00.0 "PCI bridge" "Freescale Semiconductor Inc" "Device 0440" -r20 "" ""
> > 0002:00:00.0 "PCI bridge" "Freescale Semiconductor Inc" "Device 0440" -r20 "" ""
> 
> What is the output of "virsh nodedev-dumpxml pci_0000_01_00_4"? It should
> contain an element like this:
> 
>     <capability type='phys_function'>
>     ...
> 
> This indicates that it is an sriov VF (its PF will be listed in the above <capability>
> element). If this element is missing, there is something incomplete about the
> SRIOV implementation for your card's driver.
> 
> 

It seemed that card driver is OK.

~# virsh nodedev-dumpxml pci_0000_01_00_4
<device>
  <name>pci_0000_01_00_4</name>
  <path>/sys/devices/ffe240000.pcie/pci0000:00/0000:00:00.0/0000:01:00.4</path>
  <parent>pci_0000_00_00_0</parent>
  <driver>
    <name>fslinicvf</name>
  </driver>
  <capability type='pci'>
    <domain>0</domain>
    <bus>1</bus>
    <slot>0</slot>
    <function>4</function>
    <product id='0x0000' />
    <vendor id='0x1957' />
    <capability type='phys_function'>
      <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </capability>
    <iommuGroup number='78'>
      <address domain='0x0000' bus='0x01' slot='0x00' function='0x4'/>
    </iommuGroup>
  </capability>
</device>

> >
> > # echo 1957 0000 > /sys/bus/pci/drivers/vfio-pci/new_id
> > # echo 1957 1957 > /sys/bus/pci/drivers/vfio-pci/new_id
> 
> 
> You shouldn't need to do the above when managed='yes', since libvirt does it for
> you (doing it manually like this may or may not have adverse consequences, I
> haven't tried it)

OK, I got it.

> 
> 
> >
> >     <interface type="hostdev" managed="yes">
> >       <mac address="00:e0:0c:00:20:01"/>
> >       <source>
> >         <address type="pci" domain="0x0000" bus="0x01" slot="0x00"
> function="0x04"/>
> >       </source>
> >     </interface>
> >     <memballoon model='virtio'/>
> >   </devices>
> > </domain>
> >
> > root at t4240rdb:/var/volatile# virsh start interface
> > error: Failed to start domain interface
> > error: internal error: missing IFLA_VF_INFO in netlink response
> 
> At some point (it's been quite awhile, so I've forgotten exactly when) the kernel
> stopped automatically including the IFLA_VF_INFO object in the response to a
> netlink RTM_GETLINK request. In order for this info to be included, you need to
> add an IFLA_EXT_MASK with the RTEXT_FILTER_VF bit set into the RTM_GETLINK
> request. This is done in
> src/util/virnetdev.c:virNetDevLinkDump() - here are two bugzilla records
> describing the problem and pointing at the patches added to fix it:
> 
>  https://bugzilla.redhat.com/show_bug.cgi?id=889249
>  https://bugzilla.redhat.com/show_bug.cgi?id=889319
> 
> That particular code isn't compiled into libvirt unless RTEXT_FILTER_VF is #defined,
> which should be done in /usr/include/linux/rtnetlink.h. If the kernel headers
> used when building libvirt were missing that #define (or maybe instead of it being
> #defined, it is defined as a value in an enum?), then RTEXT_FILTER_VF wouldn't
> be set in the request. Of course if RTEXT_FILTER_VF really doesn't exist, then the
> kernel should be returning IFLA_VF_INFO without needing the extra flag in the
> request.
> 
> Hopefully something in here will point you at the source of your problem.

We use linux-3.12 with RTEXT_FILTER_VF defined.
$ grep -r RTEXT_FILTER_VF ./
./include/uapi/linux/rtnetlink.h:#define RTEXT_FILTER_VF                (1 << 0)

So I guess your patches had been compiled into libvirt.

Best Regards,
Olivia