[vfio-users] No Signal: GTX 680 or GTS 450 on passthrough

Wed Aug 3 14:58:37 UTC 2016

I'd like to add a few more observations and things I have tried without
success, and if nobody has any more tests I can execute to help identify my
"no signal" problem, I'll stop bothering you all. :)

1. I noticed that in several other users' dmesg output, there's usually a
line that looks like this:
vfio-pci 0000:03:00.0: enabling device (0100->0103)
I do not see this in my dmesg under any test scenario. Is there something I
might have missed during setup that would cause vfio-pci not to try to
enable my adapter? There are no error messages either.

2. I have repeated most of my tests using a friend's GTX 970, and I got the
same result again. Flushed framebuffer, no signal. Both with and without
UEFI firmware and BIOS. At this point I'm pretty convinced my problem is
not specific to my cards.

3. I've tried disabling the VGA arbiter using the vfio-pci disable-vga
option. When I did this, I was using UEFI card firmware and guest BIOS. The
result was that qemu reported that argument x-vga was not available on the
vfio-pci device bound to the GTX 970. When I removed x-vga, I got no
signal, as before.

4. I've attempted a menagerie of host kernel arguments:
nomodeset - my host machine would not boot
video=efifb:off - no change. Not surprising, since my host BIOS is not UEFI
enabled
video=vesafb:off - dmesg reported that this argument is obsolete. No change

5. I realized that the vgaarb is always choosing the guest graphics adapter
as the boot device instead of the ASPEED host adapter. (Until vfio-pci
binds the guest adapter a second later, then vgaarb switches.) Could this
be causing trouble? Except for the disable-vga option I tried in step 3, I
was not able to find a way to change this. My host BIOS does not have an
option for boot VGA device selection. Can I force vfio-pci to bind the
guest adapter before vgaarb snatches it?

6. I've not seen a single other case of a user with an ASPEED host adapter
in all my research. Lots of AMD, NVIDIA, and Intel, though. Could I be out
of luck simply because of my host adapter is unusual?

Thanks for your time,
Robert

On Jul 24, 2016 2:45 PM, "Robert Spychalski" <rmspycha at mtu.edu> wrote:

> Greetings,
>
> Like several other people whose adventures I have reviewed on the archives
> of this list, I could use a shove in the right direction trying to get a
> VGA passthrough operational with QEMU/KVM/VFIO.
>
> The problem I am experiencing:
> When I start QEMU, my passed-through graphics card flushes—that is, if
> there was any text being displayed, it goes blank—but then outputs no
> signal to the monitor attached to it. If I interrupt the VM and start it
> again, there is no visible effect on the attached monitor until I reboot
> the host.
>
> The hardware I am using:
> Host Graphics: ASPEED Graphics (on the motherboard)
> Passthrough Graphics: GTX 680 & GTS 450 (not simultaneously. I've tried
> each separately with identical results.)
> CPUs: 2x Xeon E5-2670 (SR0KX stepping)
> Motherboard: Asus Z9PA-D8
>
> $ lspci | grep -i vga
> 03:00.0 VGA compatible controller: NVIDIA Corporation GF106 [GeForce GTS
> 450] (rev a1)
> 0c:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics
> Family (rev 21)
>
> IOMMU is enabled. The NVIDIA graphics adapter is isolated. I have not
> applied any isolation or arbitration patches, and I am not using the unsafe
> interrupt hack:
>
> $ cat /proc/cmdline
> BOOT_IMAGE=/ROOT/ubuntu@/boot/vmlinuz-4.4.0-31-generic
> root=ZFS=rpool/ROOT/ubuntu ro intel_iommu=on
> $ readlink /sys/bus/pci/devices/0000:03:00.0/iommu_group
> ../../../../kernel/iommu_groups/19
> $ ls /sys/kernel/iommu_groups/19/devices
> 0000:03:00.0  0000:03:00.1
>
> I do not have NVIDIA proprietary drivers installed. Nouveau is
> blacklisted. vfio-pci successfully binds to the passthrough graphics
> adapter and its audio function during the boot process. The first two IDs
> are for the 680; the second pair is for the 450. (Again, I don't have both
> cards in the server at the same time.)
>
> $ cat /etc/modprobe.d/vfio.conf
> options vfio-pci ids=10de:1180,10de:0e0a,10de:0dc4,10de:0be9
> $ cat /etc/modprobe.d/blacklist-nouveau.conf
> blacklist nouveau
> blacklist lbm-nouveau
> options nouveau modeset=0
> alias nouveau off
> alias lbm-nouveau off
> $ cat /etc/modprobe.d/nouveau-kms.conf
> options nouveau modeset=0
> $ lspci -nnk -s 03:00
> 03:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF106
> [GeForce GTS 450] [10de:0dc4] (rev a1)
>         Subsystem: eVga.com. Corp. GF106 [GeForce GTS 450] [3842:1450]
>         Kernel driver in use: vfio-pci
>         Kernel modules: nvidiafb, nouveau
> 03:00.1 Audio device [0403]: NVIDIA Corporation GF106 High Definition
> Audio Controller [10de:0be9] (rev a1)
>         Subsystem: eVga.com. Corp. GF106 High Definition Audio Controller
> [3842:1450]
>         Kernel driver in use: vfio-pci
>         Kernel modules: snd_hda_intel
> $ lsmod | grep -e vfio -e kvm
> kvm_intel             172032  0
> kvm                   536576  1 kvm_intel
> vfio_pci               40960  0
> irqbypass              16384  2 kvm,vfio_pci
> vfio_virqfd            16384  1 vfio_pci
> vfio_iommu_type1       20480  0
> vfio                   28672  2 vfio_iommu_type1,vfio_pci
>
> I am currently just trying to get any output at all (just bios/boot
> failure message would be fine) on my monitor attached to the passthrough
> graphics adapter. I am not using virsh, and my server is headless so I am
> restricted to using shell commands. The QEMU command I am executing follows:
>
> qemu-system-x86_64 \
>         -enable-kvm \
>         -m 1024 \
>         -cpu host,kvm=off \
>         -smp 1,sockets=1,cores=1,threads=1 \
>         -device vfio-pci,host=03:00.0,x-vga=on \
>         -device vfio-pci,host=03:00.1 \
>         -vga none \
>         -nographic
>
> Other things I have already tried:
> - I've done the whole process with both a GTS 450 and a GTX 680 (same
> result with each)
> - I have tried using an OVMF bios for QEMU and a vfio romfile argument for
> the GTX 680 that supports UEFI (same result)
> - I've used vfio argument multifunction=on (same result)
> - I've used the OVMF bios without a vfio romfile argument (QEMU goes to a
> boot failure screen with a console over ssh, still the same result on
> attached monitor)
> - I've used -nodefaults, -no-user-settings, -serial none, -parallel none,
> -nodefconfig (all same result)
> - I started the VM with a Windows.iso -cdrom (with UEFI bios
> configurations, it shows the Loading Files progress bar over ssh. With
> others, just some variable CPU utilization. With all, no signal to
> passthrough monitor)
>
> I'm completely stumped. If anyone can nudge me toward a solution from
> here, I'd be grateful.
>
> Finally, my Linux kernel is version 4.4. Here are the other software
> versions I am using:
>
> $ modinfo vfio_pci | grep version
> version:        0.2
> srcversion:     29354E84AD3D427BF06E219
> vermagic:       4.4.0-31-generic SMP mod_unload modversions
> $ modinfo kvm | grep version
> srcversion:     A3EF873CF5315736D63A5C2
> vermagic:       4.4.0-31-generic SMP mod_unload modversions
> $ qemu-system-x86_64 -version
> QEMU emulator version 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.2), Copyright (c)
> 2003-2008 Fabrice Bellard
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160803/74de95dc/attachment.htm>