[vfio-users] No Signal: GTX 680 or GTS 450 on passthrough

[Robert Spychalski]

Greetings,

Like several other people whose adventures I have reviewed on the archives
of this list, I could use a shove in the right direction trying to get a
VGA passthrough operational with QEMU/KVM/VFIO.

The problem I am experiencing:
When I start QEMU, my passed-through graphics card flushes—that is, if
there was any text being displayed, it goes blank—but then outputs no
signal to the monitor attached to it. If I interrupt the VM and start it
again, there is no visible effect on the attached monitor until I reboot
the host.

The hardware I am using:
Host Graphics: ASPEED Graphics (on the motherboard)
Passthrough Graphics: GTX 680 & GTS 450 (not simultaneously. I've tried
each separately with identical results.)
CPUs: 2x Xeon E5-2670 (SR0KX stepping)
Motherboard: Asus Z9PA-D8

$ lspci | grep -i vga
03:00.0 VGA compatible controller: NVIDIA Corporation GF106 [GeForce GTS
450] (rev a1)
0c:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics
Family (rev 21)

IOMMU is enabled. The NVIDIA graphics adapter is isolated. I have not
applied any isolation or arbitration patches, and I am not using the unsafe
interrupt hack:

$ cat /proc/cmdline
BOOT_IMAGE=/ROOT/ubuntu@/boot/vmlinuz-4.4.0-31-generic
root=ZFS=rpool/ROOT/ubuntu ro intel_iommu=on
$ readlink /sys/bus/pci/devices/0000:03:00.0/iommu_group
../../../../kernel/iommu_groups/19
$ ls /sys/kernel/iommu_groups/19/devices
0000:03:00.0  0000:03:00.1

I do not have NVIDIA proprietary drivers installed. Nouveau is blacklisted.
vfio-pci successfully binds to the passthrough graphics adapter and its
audio function during the boot process. The first two IDs are for the 680;
the second pair is for the 450. (Again, I don't have both cards in the
server at the same time.)

$ cat /etc/modprobe.d/vfio.conf
options vfio-pci ids=10de:1180,10de:0e0a,10de:0dc4,10de:0be9
$ cat /etc/modprobe.d/blacklist-nouveau.conf
blacklist nouveau
blacklist lbm-nouveau
options nouveau modeset=0
alias nouveau off
alias lbm-nouveau off
$ cat /etc/modprobe.d/nouveau-kms.conf
options nouveau modeset=0
$ lspci -nnk -s 03:00
03:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF106 [GeForce
GTS 450] [10de:0dc4] (rev a1)
        Subsystem: eVga.com. Corp. GF106 [GeForce GTS 450] [3842:1450]
        Kernel driver in use: vfio-pci
        Kernel modules: nvidiafb, nouveau
03:00.1 Audio device [0403]: NVIDIA Corporation GF106 High Definition Audio
Controller [10de:0be9] (rev a1)
        Subsystem: eVga.com. Corp. GF106 High Definition Audio Controller
[3842:1450]
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel
$ lsmod | grep -e vfio -e kvm
kvm_intel             172032  0
kvm                   536576  1 kvm_intel
vfio_pci               40960  0
irqbypass              16384  2 kvm,vfio_pci
vfio_virqfd            16384  1 vfio_pci
vfio_iommu_type1       20480  0
vfio                   28672  2 vfio_iommu_type1,vfio_pci

I am currently just trying to get any output at all (just bios/boot failure
message would be fine) on my monitor attached to the passthrough graphics
adapter. I am not using virsh, and my server is headless so I am restricted
to using shell commands. The QEMU command I am executing follows:

qemu-system-x86_64 \
        -enable-kvm \
        -m 1024 \
        -cpu host,kvm=off \
        -smp 1,sockets=1,cores=1,threads=1 \
        -device vfio-pci,host=03:00.0,x-vga=on \
        -device vfio-pci,host=03:00.1 \
        -vga none \
        -nographic

Other things I have already tried:
- I've done the whole process with both a GTS 450 and a GTX 680 (same
result with each)
- I have tried using an OVMF bios for QEMU and a vfio romfile argument for
the GTX 680 that supports UEFI (same result)
- I've used vfio argument multifunction=on (same result)
- I've used the OVMF bios without a vfio romfile argument (QEMU goes to a
boot failure screen with a console over ssh, still the same result on
attached monitor)
- I've used -nodefaults, -no-user-settings, -serial none, -parallel none,
-nodefconfig (all same result)
- I started the VM with a Windows.iso -cdrom (with UEFI bios
configurations, it shows the Loading Files progress bar over ssh. With
others, just some variable CPU utilization. With all, no signal to
passthrough monitor)

I'm completely stumped. If anyone can nudge me toward a solution from here,
I'd be grateful.

Finally, my Linux kernel is version 4.4. Here are the other software
versions I am using:

$ modinfo vfio_pci | grep version
version:        0.2
srcversion:     29354E84AD3D427BF06E219
vermagic:       4.4.0-31-generic SMP mod_unload modversions
$ modinfo kvm | grep version
srcversion:     A3EF873CF5315736D63A5C2
vermagic:       4.4.0-31-generic SMP mod_unload modversions
$ qemu-system-x86_64 -version
QEMU emulator version 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.2), Copyright (c)
2003-2008 Fabrice Bellard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160724/aa7aabd9/attachment.htm>