[vfio-users] KVM internal error. Suberror: 1, emulation failure problem

[Alex Williamson]

On Fri, 6 May 2016 23:30:02 +0800
"xuemin.wen" <Kayv at qq.com> wrote:

> thanks,Alex.
> 
> I use the same config to launch the VMs in the same time.Card with PCI address 02:00.0 and the 03:00.0 works fine, 82:00.0 and 83:00.0 are not.
> 
> I just check the BIOS settings.Configuration of PCI-e device is the same.
> 
> here is the lspci command output:
> 
>         02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)
>         Subsystem: LeadTek Research Inc. Device [107d:2762]
>         Kernel driver in use: vfio-pci
>         Kernel modules: nouveau
>         .......
>         03:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)
>         Subsystem: LeadTek Research Inc. Device [107d:2762]
>         Kernel driver in use: vfio-pci
>         Kernel modules: nouveau
>         ......
>         82:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)
>         Subsystem: LeadTek Research Inc. Device [107d:2762]
>         Kernel driver in use: vfio-pci
>         Kernel modules: nouveau
>         ......
>         83:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)
>         Subsystem: LeadTek Research Inc. Device [107d:2762]
>         Kernel driver in use: vfio-pci
>         Kernel modules: nouveau
> 
> 
>  
> I also edit the /usr/bin/qemu-kvm like this:
>     
>     #!/bin/sh
>     result="$@"
>     result=${result/02:00.0/02:00.0,x-vga=on}
>     result=${result/03:00.0/03:00.0,x-vga=on}
>     result=${result/82:00.0/82:00.0,x-vga=on}
>     result=${result/83:00.0/83:00.0,x-vga=on}
> 
>     exec /usr/bin/qemu-system-x86_64 -machine accel=kvm $result

That's an important piece of information.  What about the experiment of
using the ROM from one of the working cards, or at least verifying that
the ROMs we're seeing on the second root bridge are valid?  Are there
any invalid ROM messages in dmesg?  You can extract the ROM with:

echo 1 > /sys/bus/pci/devices/0000:02:00.0/rom
cat /sys/bus/pci/devices/0000:02:00.0/rom > GTX970.rom
echo 0 > /sys/bus/pci/devices/0000:02:00.0/rom

Then you can use a <rom file=.../> option in your XML using the same
ROM for all the cards.  I'm mostly just grasping that these emulation
failure issues often seem to occur during ROM execution so there must
be some difference in the ROM itself.  The problem may actually be that
the host system doesn't support VGA routing to the second root bridge
and you may have no choice but to drop VGA and use OVMF for this
configuration.  Thanks,

Alex