[libvirt] Error starting domain: internal error: missing IFLA_VF_INFO in netlink response

Mon Jan 12 10:09:34 UTC 2015

Hi Laine,

I'm not sure if /usr/include/linux/rtnetlink.h is necessary in the filesystem on host machine.

In our scenario, both host machine and iNIC card are both PowerPC boards.
I just checked the linux source code we used
$ grep -r RTEXT_FILTER_VF ./
./include/uapi/linux/rtnetlink.h:#define RTEXT_FILTER_VF                (1 << 0)
But we didn't install the linux header files on the PowerPC host machine.

Best Regards,
Olivia

> -----Original Message-----
> From: Yin Olivia-R63875
> Sent: Monday, January 12, 2015 5:48 PM
> To: 'Laine Stump'
> Cc: libvir-list at redhat.com
> Subject: RE: Error starting domain: internal error: missing IFLA_VF_INFO in netlink
> response
> 
> Hi Laine,
> 
> Thank you for the reply. I tried to dump the information.
> 
> > -----Original Message-----
> > From: sendmail [mailto:justsendmailnothingelse at gmail.com] On Behalf Of
> > Laine Stump
> > Sent: Thursday, January 08, 2015 11:13 PM
> > To: Yin Olivia-R63875
> > Cc: libvir-list at redhat.com
> > Subject: Re: Error starting domain: internal error: missing
> > IFLA_VF_INFO in netlink response
> >
> > On 01/08/2015 04:40 AM, Hong-Hua.Yin at freescale.com wrote:
> > > Hi Laine,
> > >
> > > Sorry to disturb you.
> > > It seemed this issue had been fixed in
> > > libvirt-1.2.2/libnl-3.2.22/linux-3.12. But
> > we still got the error on PowerPC platform.
> > > I'll appreciate if you could give any suggestion. We are not sure if
> > > any netlink
> > implementation in kernel space is missed.
> > >
> > > The scenario is a little complicated. We installed internal PF and
> > > VF kernel modules and want to use <interface type="hostdev"
> > > managed="yes">
> > syntax to start a guest domain with MAC address.
> > >
> > > # insmod fslinic.ko max_vfs=2
> > > Freescale 10 Gigabit PCI Express Network Driver fslinic 0000:01:00.0:
> > > Multiqueue Enabled: Rx Queue count = 1, Tx Queue count = 1 fslinic
> > > 0000:01:00.0: Freescale (R) 10 Gigabit Network Connection fslinic
> > > 0000:01:00.1: Multiqueue Enabled: Rx Queue count = 1, Tx Queue count
> > > =
> > > 1 fslinic 0000:01:00.1: Freescale (R) 10 Gigabit Network Connection
> > >
> > > # insmod fslinicvf.ko
> > > Freescale 10 Gigabit PCI Express Network Driver
> > >
> > > # lspci -mk
> > > 00:00.0 "PCI bridge" "Freescale Semiconductor Inc" "Device 0440" -r20 "" ""
> > > 01:00.0 "Power PC" "Freescale Semiconductor Inc" "Device 0440" -r20 "" ""
> > > 01:00.1 "Power PC" "Freescale Semiconductor Inc" "Device 0440" -r20 "" ""
> > > 01:00.4 "Power PC" "Freescale Semiconductor Inc" "Device 0000" -r20 "" ""
> > > 01:00.5 "Power PC" "Freescale Semiconductor Inc" "Device 1957" -r20 "" ""
> > > 01:01.0 "Power PC" "Freescale Semiconductor Inc" "Device 0000" -r20 "" ""
> > > 01:01.1 "Power PC" "Freescale Semiconductor Inc" "Device 1957" -r20 "" ""
> > > 0001:00:00.0 "PCI bridge" "Freescale Semiconductor Inc" "Device 0440" -r20 ""
> ""
> > > 0002:00:00.0 "PCI bridge" "Freescale Semiconductor Inc" "Device 0440" -r20 ""
> ""
> >
> > What is the output of "virsh nodedev-dumpxml pci_0000_01_00_4"? It
> > should contain an element like this:
> >
> >     <capability type='phys_function'>
> >     ...
> >
> > This indicates that it is an sriov VF (its PF will be listed in the
> > above <capability> element). If this element is missing, there is
> > something incomplete about the SRIOV implementation for your card's driver.
> >
> >
> 
> It seemed that card driver is OK.
> 
> ~# virsh nodedev-dumpxml pci_0000_01_00_4 <device>
>   <name>pci_0000_01_00_4</name>
>   <path>/sys/devices/ffe240000.pcie/pci0000:00/0000:00:00.0/0000:01:00.4</path>
>   <parent>pci_0000_00_00_0</parent>
>   <driver>
>     <name>fslinicvf</name>
>   </driver>
>   <capability type='pci'>
>     <domain>0</domain>
>     <bus>1</bus>
>     <slot>0</slot>
>     <function>4</function>
>     <product id='0x0000' />
>     <vendor id='0x1957' />
>     <capability type='phys_function'>
>       <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
>     </capability>
>     <iommuGroup number='78'>
>       <address domain='0x0000' bus='0x01' slot='0x00' function='0x4'/>
>     </iommuGroup>
>   </capability>
> </device>
> 
> > >
> > > # echo 1957 0000 > /sys/bus/pci/drivers/vfio-pci/new_id
> > > # echo 1957 1957 > /sys/bus/pci/drivers/vfio-pci/new_id
> >
> >
> > You shouldn't need to do the above when managed='yes', since libvirt
> > does it for you (doing it manually like this may or may not have
> > adverse consequences, I haven't tried it)
> 
> OK, I got it.
> 
> >
> >
> > >
> > >     <interface type="hostdev" managed="yes">
> > >       <mac address="00:e0:0c:00:20:01"/>
> > >       <source>
> > >         <address type="pci" domain="0x0000" bus="0x01" slot="0x00"
> > function="0x04"/>
> > >       </source>
> > >     </interface>
> > >     <memballoon model='virtio'/>
> > >   </devices>
> > > </domain>
> > >
> > > root at t4240rdb:/var/volatile# virsh start interface
> > > error: Failed to start domain interface
> > > error: internal error: missing IFLA_VF_INFO in netlink response
> >
> > At some point (it's been quite awhile, so I've forgotten exactly when)
> > the kernel stopped automatically including the IFLA_VF_INFO object in
> > the response to a netlink RTM_GETLINK request. In order for this info
> > to be included, you need to add an IFLA_EXT_MASK with the
> > RTEXT_FILTER_VF bit set into the RTM_GETLINK request. This is done in
> > src/util/virnetdev.c:virNetDevLinkDump() - here are two bugzilla
> > records describing the problem and pointing at the patches added to fix it:
> >
> >  https://bugzilla.redhat.com/show_bug.cgi?id=889249
> >  https://bugzilla.redhat.com/show_bug.cgi?id=889319
> >
> > That particular code isn't compiled into libvirt unless
> > RTEXT_FILTER_VF is #defined, which should be done in
> > /usr/include/linux/rtnetlink.h. If the kernel headers used when
> > building libvirt were missing that #define (or maybe instead of it
> > being #defined, it is defined as a value in an enum?), then
> > RTEXT_FILTER_VF wouldn't be set in the request. Of course if
> > RTEXT_FILTER_VF really doesn't exist, then the kernel should be returning
> IFLA_VF_INFO without needing the extra flag in the request.
> >
> > Hopefully something in here will point you at the source of your problem.
> 
> We use linux-3.12 with RTEXT_FILTER_VF defined.
> $ grep -r RTEXT_FILTER_VF ./
> ./include/uapi/linux/rtnetlink.h:#define RTEXT_FILTER_VF                (1 << 0)
> 
> So I guess your patches had been compiled into libvirt.
> 
> Best Regards,
> Olivia