[vfio-users] KVM internal error. Suberror: 1, emulation failure problem

[xuemin.wen]

hello everyone,
    I am trying to passthrough 4*GTX970 video card to individual VMs,here is the problem.

    the GTX970 with PCI address 02:00.0 and 03:00.0 can passthrough to the VMs,and works good.
    
    the GTX970 with PCI address 82:00.0 and 83:00.0 also can passthrough to the VMs,but the VMs just work for a few seconds with CPU usage 100%,and return a error:"KVM internal error. Suberror: 1 emulation failure"

    if i don't passtrough GTX 970,there is no error. So I think the issue with the relevant VFIO,not the QEMU-KVM. 
    
    my host hardware is :
        Supermicro 1028GQ-TR system include X10DGQ motherboard
        XEON E5 2630v3 * 2
        16GB RAM * 4
        LEADTEK GTX970 * 4
        INTEL SSD S3510

    my host system is :
        Fedora 22 server at 4.4.5-200.fc22.x86_64
        QEMU emulator version 2.3.1 (qemu-2.3.1-12.fc22)
        libvirt version: 1.2.13.2

    the log of GTX with PCI address 82:00.0 or 83:00.0 

2016-05-06 13:37:10.197+0000: starting up libvirt version: 1.2.13.2, package: 2.fc22 (Fedora Project, 2016-01-21-01:06:26, buildvm-06.phx2.fedoraproject.org), qemu version: 2.3.1 (qemu-2.3.1-12.fc22)
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu-kvm -name instance-000002ef -S -machine pc-i440fx-2.3,accel=kvm,usb=off -cpu Haswell-noTSX,+abm,+pdpe1gb,+rdrand,+f16c,+osxsave,+dca,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme,kvm=off -m 4096 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid 92af0144-107e-48e1-bed1-ad04fac1faea -smbios type=1,manufacturer=Fedora Project,product=OpenStack Nova,version=2014.2.4-1.fc22,serial=26348ff0-e2d1-459f-a026-ec33b800e05c,uuid=92af0144-107e-48e1-bed1-ad04fac1faea -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-000002ef.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive file=/var/lib/nova/instances/92af0144-107e-48e1-bed1-ad04fac1faea/disk,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x3,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=27,id=hostnet0,vhost=on,vhostfd=29 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:30:0d:09,bus=pci.0,addr=0x2 -chardev file,id=charserial0,path=/var/lib/nova/instances/92af0144-107e-48e1-bed1-ad04fac1faea/console.log -device isa-serial,chardev=charserial0,id=serial0 -chardev pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1 -device usb-tablet,id=input0 -device vfio-pci,host=83:00.0,id=hostdev0,bus=pci.0,addr=0x4 -device vfio-pci,host=83:00.1,id=hostdev1,bus=pci.0,addr=0x5 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -msg timestamp=on
char device redirected to /dev/pts/3 (label charserial1)
KVM internal error. Suberror: 1
emulation failure
EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000000
ESI=00000000 EDI=00000000 EBP=00000000 ESP=00006de8
EIP=0006fdff EFL=00010006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 ffffffff 00809300
CS =3020 00030200 ffffffff 00809b00
SS =0000 00000000 ffffffff 00809300
DS =0000 00000000 ffffffff 00809300
FS =0000 00000000 ffffffff 00809300
GS =0000 00000000 ffffffff 00809300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT=     000f6b70 00000037
IDT=     00000000 000003ff
CR0=00000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
Code=00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
qemu: terminating on signal 15 from pid 1500
2016-05-06 13:44:57.035+0000: shutting down
      
thanks,

xuemin.
        
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160506/392b6304/attachment.htm>