[vfio-users] RmInitAdapter failed for nvidia in ubuntu guest

[Cepheus]

Hello,

I am trying to build a gaming guest (Ubuntu, Mint, SteamOS or something
Ubuntu-LTS-based) on a gentoo host. The host is mostly gentoo amd64,
hardened kernel (but with some options disabled to not get in the way),
PIC/PIE and SSP toolchain. Relevant versions are:

kernel-4.4.8
libvirt-1.3.4
qemu-2.5.1

The UEFI image is from here, as of 2016-06-24:
https://www.kraxel.org/repos/jenkins/edk2/

Two nvidia graphics cards: The passive one (no fan) is a GeForce GTX 750
Ti. Used by the host. The active one - a GeForce GTX 750 - is "hidden"
from the host via pci-stub.ids boot parameter and assigned to the guest
(both the video and the audio device, which share one IOMMU group, with
no ther devices in the group). The card is UEFI capable, according to
the test program from here:
http://vfio.blogspot.de/2014/08/does-my-graphics-card-rom-support-efi.html

I can install Ubuntu 16.04 fine and it works, as long as it uses the
nouveau driver. The nvidia driver is listed in "Additional drivers" as
compatible and can be installed, but on guest reboot the X server cannot
use the card.

These dmesg lines from the guest appear relevant:

[    4.948877] NVRM: RmInitAdapter failed! (0x23:0x56:464)
[    4.949354] NVRM: rm_init_adapter failed for device bearing minor
number 0
<repeated 12 times>

Xorg.0.log says:

[     9.285] (EE) NVIDIA(GPU-0): The NVIDIA GPU at PCI:0:2:0 is not
supported by the 361.42
[     9.285] (EE) NVIDIA(GPU-0):     NVIDIA driver.
[     9.285] (EE) NVIDIA(GPU-0): Failed to initialize the NVIDIA
graphics device!

Full logs are linked below.

I am sure driver versions 361.* support GPUs in the 700 range. The host
has nearly the same configuration with nvidia-drivers-361.28.

When I try to start the guest with "nomodeset" kernel parameter, the
host hangs. Not even Alt+PrintScrn+REISUB is possible.

CPU is a Core i7 5930K, on a X99A mainboard (MSI X99A Gaming 7).

Before I start aimlessly changing things: Does anyone have an idea how
to make it work, with nvidia-drivers in the guest?

Things I could try:
- update kernel
- update libvirt
- update qemu
- guest boot options: acpi=off, noacpi, noapic, nolapic, vmalloc=xM, ...?
- switch to non-hardened host kernel
- switch to non-hardened host toolchain (will take a while to rebuild
everything)
- try other distributions as guest
- other version of guest nvidia driver
- switch the graphics card for the guest

-
Guest dmesg: https://bpaste.net/show/719a56868947
Guest lsmod -k: https://bpaste.net/show/d9b6dc11bf71
Guest lspci: https://bpaste.net/show/18d489d5f77c
Guest Xorg.0.log: https://bpaste.net/show/e19d4bd301a3
Guest configuration: https://bpaste.net/show/b2862886ee84
Host kernel config: https://bpaste.net/show/7536e18f0131
-


Many thanks!

Kai