[vfio-users] vfio fails Guest FreeBSD9.3 host Fedora 23
chintu hetam
rometoroam at gmail.com
Fri May 27 15:57:38 UTC 2016
Hi Alex,
Thank you for the quick response.
i have 370+G RAM in the system
[root at localhost vcr]# cat /proc/meminfo
MemTotal: 396231416 kB
MemFree: 272301316 kB
the reason i moved towards pci-e, sr-iov, vfio is primarily that reason. i
can get upto 1.8Gbps iMix traffic performance on vhost-virtio-passthrough
mode, i am looking for upwards to 4Gbps performance per vNIC.
the error doesn't say what;s wrong,with my config.
appreciate your help.
-chintu-
On Fri, May 27, 2016 at 11:49 AM, Alex Williamson <
alex.williamson at redhat.com> wrote:
> On Fri, 27 May 2016 11:15:41 -0400
> chintu hetam <rometoroam at gmail.com> wrote:
>
> > I have tried few things with libvirt, directly spawning VM with qemu-kvm
> > even tried withe centos7 as a guest and in all cases vfio fails.
> >
> > here are the details!!
> >
> > 1) Device: Intel X710/XL710
> > 2) virt xml snapshot:
> >
> > <hostdev mode='subsystem' type='pci' managed='yes'>
> > <driver name='vfio'/>
> > <source>
> > <address domain='0x0000' bus='0xaa' slot='0x02' function='0x0'/>
> > </source>
> > <alias name='hostdev0'/>
> > <rom bar='on'/>
> > <address type='pci' domain='0x0000' bus='0x00' slot='0x09'
> > function='0x0'/>
> > </hostdev>
> > <hostdev mode='subsystem' type='pci' managed='yes'>
> > <driver name='vfio'/>
> > <source>
> > <address domain='0x0000' bus='0xaa' slot='0x06' function='0x0'/>
> > </source>
> > <alias name='hostdev1'/>
> > <rom bar='on'/>
> > <address type='pci' domain='0x0000' bus='0x00' slot='0x0b'
> > function='0x0'/>
> > </hostdev>
> >
> > 3) ip link show snapshot:
> > 18: enp170s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop portid
> > 00e0ed5d884c state DOWN mode DEFAULT group default qlen 1000
> > link/ether 00:e0:ed:5d:88:4c brd ff:ff:ff:ff:ff:ff
> > vf 0 MAC aa:bb:cc:dd:ee:ff, spoof checking on, link-state auto
> > vf 1 MAC aa:bb:cc:dd:ee:fe, spoof checking on, link-state auto
> > vf 2 MAC aa:bb:cc:dd:ee:fd, spoof checking on, link-state auto
> > vf 3 MAC aa:bb:cc:dd:ee:fc, spoof checking on, link-state auto
> > 19: enp170s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop portid
> > 00e0ed5d884d state DOWN mode DEFAULT group default qlen 1000
> > link/ether 00:e0:ed:5d:88:4d brd ff:ff:ff:ff:ff:ff
> > vf 0 MAC aa:bb:cc:dd:ed:ff, spoof checking on, link-state auto
> > vf 1 MAC aa:bb:cc:dd:ed:fe, spoof checking on, link-state auto
> > vf 2 MAC aa:bb:cc:dd:ed:fd, spoof checking on, link-state auto
> > vf 3 MAC aa:bb:cc:dd:ed:fc, spoof checking on, link-state auto
> >
> > 4) various system information
> > [root at localhost vcr]# modinfo i40evf
> > filename:
> >
> /lib/modules/4.4.6-300.fc23.x86_64/updates/drivers/net/ethernet/intel/i40evf/i40evf.ko
> > version: 1.5.14
> > license: GPL
> > description: Intel(R) 40-10 Gigabit Ethernet Virtual Function Driver
> > author: Intel Corporation, <linux.nics at intel.com>
> > srcversion: 522158BAB323575B03D48C8
> > alias: pci:v00008086d0000154Csv*sd*bc*sc*i*
> > depends:
> > vermagic: 4.4.6-300.fc23.x86_64 SMP mod_unload
> >
> > [root at localhost vcr]# modinfo i40e
> > filename:
> >
> /lib/modules/4.4.6-300.fc23.x86_64/kernel/drivers/net/ethernet/intel/i40e/i40e.ko.xz
> > version: 1.3.46-k
> > license: GPL
> > description: Intel(R) Ethernet Connection XL710 Network Driver
> > author: Intel Corporation, <e1000-devel at lists.sourceforge.net>
> > srcversion: A3FBA4AB3B91C574EA1FD38
> > alias: pci:v00008086d00001588sv*sd*bc*sc*i*
> > alias: pci:v00008086d00001587sv*sd*bc*sc*i*
> > alias: pci:v00008086d000037D2sv*sd*bc*sc*i*
> > alias: pci:v00008086d000037D1sv*sd*bc*sc*i*
> > alias: pci:v00008086d000037D0sv*sd*bc*sc*i*
> > alias: pci:v00008086d00001587sv*sd*bc*sc*i*
> > alias: pci:v00008086d00001589sv*sd*bc*sc*i*
> > alias: pci:v00008086d00001586sv*sd*bc*sc*i*
> > alias: pci:v00008086d00001585sv*sd*bc*sc*i*
> > alias: pci:v00008086d00001584sv*sd*bc*sc*i*
> > alias: pci:v00008086d00001583sv*sd*bc*sc*i*
> > alias: pci:v00008086d00001581sv*sd*bc*sc*i*
> > alias: pci:v00008086d00001580sv*sd*bc*sc*i*
> > alias: pci:v00008086d0000157Fsv*sd*bc*sc*i*
> > alias: pci:v00008086d00001574sv*sd*bc*sc*i*
> > alias: pci:v00008086d00001572sv*sd*bc*sc*i*
> > depends: ptp,vxlan
> > intree: Y
> > vermagic: 4.4.6-300.fc23.x86_64 SMP mod_unload
> > parm: debug:Debug level (0=none,...,16=all) (int)
> >
> > However, in libvirt xml if i change driver to kvm it works but then as i
> > understand it's e1000 equivalent and vfio is virtio performance
> equivalent .
>
> Not remotely, in this context 'kvm' vs 'vfio' are both types of device
> assignment, but kvm is deprecated here and will be going away at some
> point (only legacy kvm device assignment, not the remainder of kvm). If
> you want virtio performance equivalent, use virtio, PCI device
> assignment offers better performance than virtio.
>
> > 5) lspci -tvv snapshot
> >
> >
> > | +-03.0-[aa-ab]--+-00.0 Intel Corporation Ethernet
> Controller
> > X710 for 10GbE SFP+
> > | | +-00.1 Intel Corporation Ethernet
> Controller
> > X710 for 10GbE SFP+
> > | | +-00.2 Intel Corporation Ethernet
> Controller
> > X710 for 10GbE SFP+
> > | | +-00.3 Intel Corporation Ethernet
> Controller
> > X710 for 10GbE SFP+
> > | | +-02.0 Intel Corporation XL710/X710 Virtual
> > Function
> > | | +-02.1 Intel Corporation XL710/X710 Virtual
> > Function
> > | | +-02.2 Intel Corporation XL710/X710 Virtual
> > Function
> > | | +-02.3 Intel Corporation XL710/X710 Virtual
> > Function
> > | | +-06.0 Intel Corporation XL710/X710 Virtual
> > Function
> > | | +-06.1 Intel Corporation XL710/X710 Virtual
> > Function
> > | | +-06.2 Intel Corporation XL710/X710 Virtual
> > Function
> > | | \-06.3 Intel Corporation XL710/X710 Virtual
> > Function
> >
> > 6) FreeBSD guest virtual function driver version is 1.2.11
> >
> > 7) error log doesn't say much other than resource unavailable
> > 2016-05-27 15:08:28.804+0000: starting up libvirt version: 1.2.18.2,
> > package: 3.fc23 (Fedora Project, 2016-03-17-21:40:48,
> > buildvm-22.phx2.fedoraproject.org), qemu version: 2.4.1
> (qemu-2.4.1-9.fc23)
> > LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
> > QEMU_AUDIO_DRV=spice /usr/bin/qemu-kvm -name generic -S -machine
> > pc-i440fx-2.4,accel=kvm,usb=off,vmport=off,mem-merge=off -cpu SandyBridge
> > -m 32768 -mem-prealloc -mem-path /dev/hugepages/libvirt/qemu -realtime
> > mlock=on -smp 8,sockets=8,cores=1,threads=1 -uuid
> > d8f61bc1-dd9a-470f-a26f-824e5e2b8359 -no-user-config -nodefaults -chardev
> >
> socket,id=charmonitor,path=/var/lib/libvirt/qemu/generic.monitor,server,nowait
> > -mon chardev=charmonitor,id=monitor,mode=control -rtc
> > base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet
> > -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1
> > -boot order=c,menu=on,strict=on -device
> > ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device
> >
> ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6
> > -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1
> > -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2
> > -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive
> >
> file=/home/vcr/Appliance-4.6.0.0_5-vtnet0.img,if=none,id=drive-virtio-disk0,format=raw
> > -device
> >
> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x8,drive=drive-virtio-disk0,id=virtio-disk0
> > -drive
> >
> file=/dev/disk/by-id/scsi-36001405fdc0e99da3aa4b37b1403085c,if=none,id=drive-virtio-disk1,format=raw
> > -device
> >
> virtio-blk-pci,scsi=off,bus=pci.0,addr=0xa,drive=drive-virtio-disk1,id=virtio-disk1
> > -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=25 -device
> >
> virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:30:a0:c0,bus=pci.0,addr=0x3
> > -chardev pty,id=charserial0 -device
> > isa-serial,chardev=charserial0,id=serial0 -chardev
> > spicevmc,id=charchannel0,name=vdagent -device
> >
> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0
> > -spice
> >
> port=5900,addr=127.0.0.1,disable-ticketing,image-compression=off,seamless-migration=on
> > -device
> >
> qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2
> > -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device
> > hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev
> > spicevmc,id=charredir0,name=usbredir -device
> > usb-redir,chardev=charredir0,id=redir0 -chardev
> > spicevmc,id=charredir1,name=usbredir -device
> > usb-redir,chardev=charredir1,id=redir1 -device
> > vfio-pci,host=aa:02.0,id=hostdev0,bus=pci.0,addr=0x9,rombar=1 -device
> > vfio-pci,host=aa:06.0,id=hostdev1,bus=pci.0,addr=0xb,rombar=1 -device
> > virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on
> > char device redirected to /dev/pts/2 (label charserial0)
> > qemu: qemu_thread_create: Resource temporarily unavailable
> > 2016-05-27 15:09:20.735+0000: shutting down
> >
> > Any help will be appreciated.
>
> You're creating a 32G VM, how much memory does the host have? Anything
> in dmesg? Thanks,
>
> Alex
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160527/565ea634/attachment.htm>
More information about the vfio-users
mailing list