[vfio-users] vfio pass through only works once after reboot - more tests

[Alex Williamson]

On Thu, 2015-09-17 at 22:56 +0200, Karsten Elfenbein wrote:
> Hi,
> 
> after my first tries in
> http://lists.nongnu.org/archive/html/qemu-discuss/2015-09/msg00003.html
> I did an other round of tests.
> 
> I made the GF460 the primary card and put the GF970 into vfio. With
> that config the VM could start and use the card multiple times without
> rebooting. (vfio is chowned to me)
> 
> The only bad output I get from qemu while running the gf970 in the VM is:
> qemu-system-x86_64: vfio_pci_write_config(0000:06:00.0, 0x418,
> 0xe0000, 0x4) failed: Bad address
> 
> Then I switched back to the GF460 for the VM and get some more of these lines:
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x13c, 0x4)
> failed: Bad address

Hmm, "Bad address" is -EFAULT from the kernel.  We can get that if the
offset is outside the size of config space for the device, copying
faults to/from userspace, or if we do an unaligned access.  Are you
trying to use q35?  I don't know how else we'd get config accesses above
0xff.  Can you send lspci -v for the GPUs and lspci -tv for the system?

> qemu-system-x86_64: vfio_pci_write_config(0000:03:00.0, 0x13c,
> 0x3ffffff2, 0x4) failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x150, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_write_config(0000:03:00.0, 0x150,
> 0xfffffff2, 0x4) failed: Bad address
> qemu-system-x86_64: vfio_pci_write_config(0000:03:00.0, 0x48c,
> 0x118008c, 0x4) failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x45c, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_write_config(0000:03:00.0, 0x45c,
> 0xfffffff2, 0x4) failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x158, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_write_config(0000:03:00.0, 0x158,
> 0xfffffff2, 0x4) failed: Bad address
> qemu-system-x86_64: vfio_pci_write_config(0000:03:00.0, 0x610, 0x5,
> 0x4) failed: Bad address
> qemu-system-x86_64: vfio_pci_write_config(0000:03:00.0, 0x418,
> 0xe0000, 0x4) failed: Bad address
> The second boot looks the same but the card does not bring a signal.
> (not even the card bios comes up)
> 
> Installing the Nvidia driver with the GF460 crashed the Windows 10 VM
> with more if these lines:
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x60c, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x610, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x138, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_write_config(0000:03:00.0, 0x138,
> 0xfffffff2, 0x4) failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x460, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x460, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x150, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x18c, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x4a8, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x4a4, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x138, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_write_config(0000:03:00.0, 0x138,
> 0xf007fff2, 0x4) failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x460, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_write_config(0000:03:00.0, 0x460,
> 0xfffffff2, 0x4) failed: Bad address
> qemu-system-x86_64: vfio_pci_write_config(0000:03:00.0, 0x138,
> 0xfffffff2, 0x4) failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x4a8, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x4a4, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x4a8, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x4a4, 0x4)
> failed: Bad address
> qemu-system-x86_64: vfio_pci_read_config(0000:03:00.0, 0x4a8, 0x4)
> failed: Bad address
> 
> 
> Just wondering if it is possible to get the card running or better to
> just grab a GF950.

That seems like a nice upgrade.  If the 460 is all the graphics you
need, I'd be tempted to move to a 750 just for the better power
efficiency.