[libvirt-users] cannot start domain with SR-IOV interface

[Laine Stump]

On 10/15/2012 06:29 PM, Radhika Niranjan wrote:
> Hi,
>
> I've been using libvirt (versions 0.9.11.5, and recently upgraded to
> 0.10.2), and have been attaching an SR-IOV VF to my VM with the
> following device configuration:
> <hostdev mode='subsystem' type='pci' managed='yes'>
>       <source>
>         <address domain='0x0000' bus='0x07' slot='0x10' function='0x0'/>
>       </source>
>       <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
> function='0x0'/>
>  </hostdev>
>
> I had no problems with bringing up the VM and using the VF with this
> configuration.
>
> Recently I noticed that I would be able to assign a mac address to the
> SR-IOV VF if I used the interface configuration and changed over to using:
> <interface type='hostdev'>

Since your <hostdev> used managed='yes', you should use it here too:

  <interface type='hostdev' managed='yes'>

>       <mac address='52:54:00:d3:23:4f'/>
>       <source>
>         <address type='pci' domain='0x0000' bus='0x07' slot='0x10'
> function='0x0'/>
>       </source>
>       <model type='virtio'/>

<model> is unused for interface type='hostdev', so that line is
harmless, but also meaningless.
(actually it should probably log an error when it sees that, but
currently just ignores it).

>       <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
> function='0x0'/>
>     </interface>
>
> My VM fails to start up with this interface configuration. The error
> reported is:
> error: Failed to start domain exp85-1
> error: An error occurred, but the cause is unknown

This is a bit odd. It means that some code path has returned an error
without logging an error message. I tried removing "managed='yes'" and
adding <model type='virtio'/> to a similar interface definition on the
one machine where I have an SR-IOV card, and it failed to start, but
gave a slightly less meaningless message:

  error: Unable to read from monitor: Connection reset by peer

(This is less meaningless because that generally indicates that the
qemu-kvm process encountered an error and exited, so I know I can look
in /var/log/libvirt/qemu/$guest.log for an error. And indeed, when I
look there I see this:

  Failed to assign device "hostdev0" : Device or resource busy
  qemu-kvm: -device
pci-assign,host=03:10.1,id=hostdev0,configfd=32,bus=pci.0,addr=0x7:
Device 'pci-assign' could not be initialized

which makes sense.

>
> When I look into /var/log/messages I see the following messages:
>
> Oct 15 15:25:48 dcswitch85 libvirtd[17709]: 2012-10-15
> 22:25:48.145+0000: 17713: error : virFileReadAll:457 : Failed to open
> file '/var/run/libvirt/qemu/p1p1_vf0': No such file or directory

That file is used to save the previous MAC address and vlan tag of the
VF for restoration when the guest is finished with it. I'm not sure why
that error wasn't propogated back to virsh. (btw, this error isn't the
root error anyway - it would only be issued if a failure happened while
attempting to save the old MAC address/vlan of the VF. Perhaps there's a
failure in that bit of code that fails without logging an error.)

How handy are you at running gdb? If you could install the debuginfo for
libvirt, then attach gdb and set a breakpoint on the function
qemuDomainHostdevNetConfigReplace then single step, I'm pretty sure
whatever failure you're encountering is somewhere within one of the
functions called by that function. Otherwise we can probably find it by
manual examination, but that will take *much* more effort/time.

> Oct 15 15:25:48 dcswitch85 libvirtd[17709]: 2012-10-15
> 22:25:48.146+0000: 17713: error : qemuRemoveCgroup:740 : internal
> error Unable to find cgroup for exp85-1
> Oct 15 15:25:48 dcswitch85 libvirtd[17709]: 2012-10-15
> 22:25:48.146+0000: 17713: warning : qemuProcessStop:4136 : Failed to
> remove cgroup for exp85-1
>
> I was wondering if anyone else has faced this problem or has a
> solution to this.
>
> I'm using Fedora17, KVM, and an 82599EB Intel NIC.
>
> Thanks much in advance.
>
> With best regards
> Radhika
>
>
>
>
>
>
> _______________________________________________
> libvirt-users mailing list
> libvirt-users at redhat.com
> https://www.redhat.com/mailman/listinfo/libvirt-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libvirt-users/attachments/20121024/780de40b/attachment.htm>