[libvirt-users] automatically detaching PCI host devices from guest for SRIOV usage.

Tue Aug 21 04:04:24 UTC 2012

On 08/21/2012 11:22 AM, Rajesh Kumar Mallah wrote:
>
> Dear Alex / List  ,
>
> Thanks for the response.
>
> The problem is regarding detaching from the guest not host.

I see, if you attach a PCI device into guest with 'managed' 
mode(managed='yes') then the device is automatically detached from guest 
and returned to host after guest exiting, for example.

# lspci | grep Eth
00:19.0 Ethernet controller: Intel Corporation 82566DM-2 Gigabit Network 
Connection (rev 02)

# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/e1000e

# virsh start foo
Domain foo started

# ping -c 3 xx.xx.xx.xx
PING xx.xx.xx.xx (xx.xx.xx.xx) 56(84) bytes of data.
64 bytes from nx-in-f160.1e100.net (xx.xx.xx.xx): icmp_seq=1 ttl=49 
time=87.8 ms
64 bytes from nx-in-f160.1e100.net (xx.xx.xx.xx): icmp_seq=2 ttl=49 
time=85.2 ms
64 bytes from nx-in-f160.1e100.net (xx.xx.xx.xx): icmp_seq=3 ttl=49 
time=91.8 ms
--- xx.xx.xx.xx ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2093ms
rtt min/avg/max/mdev = 85.240/88.327/91.881/2.731 ms

# cat hostdev.xml
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x00' slot='0x19' function='0x0'/>
</source>
</hostdev>

# virsh attach-device foo hostdev.xml
Device attached successfully

# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/pci-stub

# ping -c 3 xx.xx.xx.xx
connect: Network is unreachable

# virsh destroy foo
Domain foo destroyed

# readlink /sys/bus/pci/devices/0000\:00\:19.0/driver/ -f
/sys/bus/pci/drivers/e1000e

# ping -c 3 xx.xx.xx.xx
PING xx.xx.xx.xx (xx.xx.xx.xx) 56(84) bytes of data.
64 bytes from xx.xx.xx.xx: icmp_seq=1 ttl=49 time=79.7 ms
64 bytes from xx.xx.xx.xx: icmp_seq=2 ttl=49 time=83.2 ms
64 bytes from xx.xx.xx.xx: icmp_seq=3 ttl=49 time=84.5 ms

--- xx.xx.xx.xx ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2087ms
rtt min/avg/max/mdev = 79.714/82.480/84.513/2.040 ms

Notes, need to wait for local network recovery again then check network 
connectivity.

If your network still is disconnect, it may be a bug IMHO, please show 
your libvirt
and qemu-kvm version, or my example is not what you want, please let me 
know.

Thanks,
Alex

> In our case since ixgbevf driver is NOT present in the host hence the 
> virtual
> interfaces never really attach themselves to the host.
>
> Regards,
> Rajesh Kumar Mallah
> Tel. 91-11-46710500 (Ext:303)
> Cell +919811255597
> Website: http://www.tradeindia.com/
> \|/ \|/ \|/   stop printing   \|/ \|/ \|/
>  |   |   |    start planting   |   |   |
>
>
> ----- "Alex Jia" <ajia at redhat.com> wrote:
> | From: "Alex Jia" <ajia at redhat.com>
> | To: "Rajesh Kumar Mallah" <mallah at tradeindia.com>
> | Cc: libvirt-users at redhat.com
> | Sent: Tuesday, August 21, 2012 8:31:21 AM
> | Subject: Re: [libvirt-users] automatically detaching PCI host 
> devices from guest for SRIOV usage.
> |
> | On 08/20/2012 08:34 PM, Rajesh Kumar Mallah wrote:
>
>     Dear List ,
>
>     We are using intel 82599EB based SRIOV capable 10GbE cards
>     on the host machine.The VFs are being assigned to the guest
>     in pass through mode and are providing the network connectivity
>     as expected.
>
>     However when we shutdown the guest the connectivity does not
>     comes back via the VFs on the guest in next boot unless
>     the VFs are detached from the *guest* *before* initiating a
>     shutdown of the guests.
>
> Dear Rajesh,
> |
> | Please see the following comments.
> |
> |
>
>     (
>     the detach is explicitly done via virsh command
>     /usr/local/libvirt/bin/virsh  detach-device guest01   ~/nic.xml
>     nic.xml contained
>     <hostdev mode='subsystem' type='pci' managed='yes'>
>         <source>
>            <address domain='0x0000' bus='0x04' slot='0x10' function='0x0'/>
>         </source>
>         <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
>     </hostdev>
>     )
>
>     The question is anyone else facing similar problem and
>     if so what solution was used?
>
>     Secondly , is there anyway to automate the detaching of the
>     VFs from the guest when the guest is put in the shutdown
>     sequence ? (or for that matter destroyed).
>
>
> | When *|managed|* is *"yes"* for a PCI device, it is automatically 
> detached from the host before being passed on to the guest, and 
> automatically *reattached* to the host after the *guest exits* such as 
> shutdown a guest.
> |
> | Regards,
> | Alex
> |
> |
>
>     Regards,
>     Rajesh Kumar Mallah
>     Tel. 91-11-46710500 (Ext:303)
>     Cell +919811255597
>     Website:http://www.tradeindia.com/
>     \|/ \|/ \|/   stop printing   \|/ \|/ \|/
>       |   |   |    start planting   |   |   |
>
>     _______________________________________________
>     libvirt-users mailing list
>     libvirt-users at redhat.com
>     https://www.redhat.com/mailman/listinfo/libvirt-users
>
>
> |

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libvirt-users/attachments/20120821/23252938/attachment.htm>