[vfio-users] Binding Radeon R9 380X 4gb to the vfio driver hangs system on recent kernels.

[wermaan at gmail.com]

After some time I found that it's not a hard freeze... I misinterpreted it
for a long time, because keyboard didn't react on caps or num key and image
was frozen. Actually I found that system is working and I can execute
commands, so I got output of dmesg and lspci after the binding and it seems
that binding succeed.

lspci:

01:00.0 0300: 1002:6938 (rev f1) (prog-if 00 [VGA controller])
    Subsystem: 1458:22c8
    Kernel driver in use: vfio-pci
    Kernel modules: amdgpu

01:00.1 0403: 1002:aad8
    Subsystem: 1458:aad8
    Kernel driver in use: vfio-pci
    Kernel modules: snd_hda_intel

02:00.0 0300: 10de:104a (rev a1) (prog-if 00 [VGA controller])
    Subsystem: 1458:362a
    Kernel driver in use: nvidia
    Kernel modules: nouveau, nvidia_drm, nvidia

dmesg:

[  338.322244] vgaarb: device changed decodes:
PCI:0000:01:00.0,olddecodes=io+mem,decodes=io+mem:owns=io+mem

And if I try to launch vm I also get in systemctl:

kernel: vgaarb: device changed decodes:
PCI:0000:01:00.0,olddecodes=io+mem,decodes=io+mem:owns=io+mem
kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19 at 0x270
kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x1b at 0x2d0
kernel: pmd_set_huge: Cannot satisfy [mem 0xb0000000-0xb0200000] with a
huge-page mapping due to MTRR override.

amd drivers are blacklisted, so there is no drivers associated with gpu
before binding to the vfio as I see from lspci.

And I've got one more thought - while nvidia gpu has nvidia driver loaded,
desktop enviroment isn't rendered via it, but via "Gallium 0.4 on llvmpipe
(LLVM 3.8, 128 bits)" as stated in "details" of gnome. So after binding amd
gpu to the vfio-pci something starting to going wrong with video output.

So I've deleted nvidia driver (tried to use nouveau but failed to do this
atm.) and binded amd gpu to the pci_stub, after that I was able to boot vm,
but it seems it shares output with desktop (I don't even know what's going
on there), because on other monitor input there is no image and I was able
to switch from vm to linux desktop once. After all I think there is no
issue with vfio here, but strange behaviour of nvidia drivers and my wrong
configuration (I still don't know how to make even nouveau drivers work...).

I think this issue can be closed.

And thank you for your answer!

werman

2016-06-25 20:41 GMT+03:00 Zir Blazer <zir_blazer at hotmail.com>:

> If it hard freezes when executing the vfio bind script for the Radeon, its
> probable than the crash happens either when it gets unbinded from the
> radeon Driver, or attached to vfio-pci in a dirty state. I don't see it
> blacklisted anywhere, so try doing so so it doesn't get initialized and try
> again. If you're using only the GeForce for the host, you aren't missing
> anything by doing so.
>
> Add this to the Kernel Parameters:
>
> modprobe.blacklist=radeon
>
> If that works, I'm curious why the behavior change with the new Kernel...
>
> _______________________________________________
> vfio-users mailing list
> vfio-users at redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160625/3de76ab9/attachment.htm>