[libvirt-users] SR-IOV Pool Availability

[Laine Stump]

On 09/20/2013 02:28 AM, Dax Kelson wrote:
> I have an Intel X540 SR-IOV NIC in a box with ACS support running RHEL6.4.
>
> Hi Hongming, I'm hoping you can please help with two issues:
>
> 1. The pool works, but I transparent VLAN with the pool does not. I
> have to manually run "ip" afterwards to get it to work.
>
> #  virsh net-dumpxml em2net
> <network connections='1'>
>   <name>em2net</name>
>   <uuid>d5dabeb2-fa09-3edf-660a-db42d8cd3f4d</uuid>
>   <forward mode='hostdev' managed='yes'>
>     <pf dev='em2'/>
>       <snip giant list of VFs>
>   </forward>
> </network>
>
> #  virsh dumpxml guestbox
> ---
>     <interface type='network'>
>       <mac address='52:54:00:c0:de:00'/>
>       <source network='em2net'/>
>       <vlan>
>         <tag id='17'/>
>       </vlan>
>       <alias name='hostdev0'/>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
> function='0x0'/>
>     </interface>
> ---
>
> The networking doesn't work until I run the following command on the host:
>
> ip link set em2 vf 0 vlan 17
>
> Then networking in the guest starts working and the following shows up
> in dmesg on the host:
>
> ixgbe 0000:01:00.1: Setting VLAN 17, QOS 0x0 on VF 0

This is a known bug in RHEL6.4, which will be fixed in RHEL6.5:

  https://bugzilla.redhat.com/show_bug.cgi?id=999107

In the meantime, you should be able to work around the bug by adding the
same <vlan> element to each guest's <interface> (if that also doesn't
work, then you are also suffering from a bug caused by a change to the
kernel NETLINK ABI - in that case you should update kernel, libnl, and
libvirt packages (at least) to the latest 6.4 versions available.)

If the workaround isn't adequate for you, you should open a ticket with
Red Hat support to get the fix for that bug backported to RHEL6.4 libvirt.

>
> 2. I can only do a single SR-IOV NIC per guest. If I try to add
> another NIC (or even PCI passthrough), I get "Duplicate ID 'hostdev0'
> for device"

This is also a known bug in RHEL6.4:

   https://bugzilla.redhat.com/show_bug.cgi?id=1001881

Again, if it is causing you problems please open a ticket with Red Hat
support so that the bug can be reprioritized.