[vfio-users] can't get my video card to connect to my vm

[Stein van Broekhoven]

I tried 2 things to avoid: "fglrx driver claiming the interrupt"
Start vm before fglrx gets to claiming it. (bad idea!)

Edit xorg related config to see what is connecting the driver to the card. (better idea)
All i found was PCI:1:0:0 (first GPU) in the config
Also tried "# aticonfig --adapter=0 --initial" and got the same config as my system created for me.

Any ideas on how i can keep the following process from claiming it?
dmesg:
[   12.361476] <6>[fglrx] Firegl kernel thread PID: 659
[   12.361547] <6>[fglrx] Firegl kernel thread PID: 660
[   12.361616] <6>[fglrx] Firegl kernel thread PID: 661
[   12.361730] <6>[fglrx] IRQ 55 Enabled
[   12.439300] <6>[fglrx] Reserved FB block: Shared offset:0, size:1000000 
[   12.439302] <6>[fglrx] Reserved FB block: Unshared offset:fab4000, size:4000 
[   12.439303] <6>[fglrx] Reserved FB block: Unshared offset:fab8000, size:548000 

ps faux:
root         2  0.0  0.0      0     0 ?        S    22:53   0:00 [kthreadd]
...
root       659  0.0  0.0      0     0 ?        S    22:53   0:00  \_ [firegl]
...



Recap:
Dear All,

I can't get the video card which is using vfio to connect to my vm:
lspci -nnk
06:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Turks XT [Radeon HD 6670/7670] [1002:6758]
        Subsystem: ASUSTeK Computer Inc. Device [1043:03ea]
        Kernel driver in use: vfio-pci
        Kernel modules: radeon, fglrx
06:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Turks/Whistler HDMI Audio [Radeon HD 6000 Series] [1002:aa90]
        Subsystem: ASUSTeK Computer Inc. Device [1043:aa90]
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel

I start the vm from cli with qemu command with sudo
The devices 06:00.0 and 06:00.1 are together in group 22
I added /dev/vfio/22 to acl in qemu config

When using vfio_iommu_type1.allow_unsafe_interrupts=1 on start vm I get the error:
stdout:
qemu-system-x86_64: -device vfio-pci,host=06:00.0,addr=09.0,multifunction=on: vfio: Error: Failed to setup INTx fd: Device or resource busy
qemu-system-x86_64: -device vfio-pci,host=06:00.0,addr=09.0,multifunction=on: Device initialization failed
dmesg:
genirq: Flags mismatch irq 55. 00000080 (vfio-int0000:06:00.0)) vs. 00000800 (fglrx[1]@PCI:6:0:0)

Without vfio_iommu_type1.allow_unsafe_interrupts=1 on start vm I get the error:
stdout:
vfio: failed to set iommu for container: Operation not permitted 
dmesg:
vfio_iommu_type1_attach_group: No interrupt remapping support.  Use the module param "allow_unsafe_interrupts" to enable VFIO IOMMU support on this platform

system:
sabertooth 990FX v1.1
AMD FX-8120
main GPU: Radeon HD 5xxx 
VM GPU: Radeon HD 6xxx
Manjaro (arch) kernel 4.1


----- Oorspronkelijk bericht -----
Van: "Alex Williamson" <alex.l.williamson at gmail.com>
Aan: "Stein van Broekhoven" <stein at aapjeisbaas.nl>
Cc: "vfio-users" <vfio-users at redhat.com>
Verzonden: Maandag 7 maart 2016 19:47:05
Onderwerp: Re: [vfio-users] can't get my video card to connect to my vm

On Mon, Mar 7, 2016 at 11:22 AM, Stein van Broekhoven < stein at aapjeisbaas.nl > wrote: 



Thanks that got me a bit further. 
dmesg | grep AMD-Vi 
[ 0.233077] [Firmware Bug]: AMD-Vi: IOAPIC[9] not in IVRS table 
[ 0.233083] [Firmware Bug]: AMD-Vi: IOAPIC[10] not in IVRS table 
[ 0.233087] [Firmware Bug]: AMD-Vi: No southbridge IOAPIC found 
[ 0.233090] AMD-Vi: Disabling interrupt remapping 
[ 1.901086] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40 
[ 1.917421] AMD-Vi: Lazy IO/TLB flushing enabled 

Any starting point where I can find info on how to get iommu remapping in my kernel? 




Firmware bug... BIOS update? This is not blocking you though, it just means you need to use the allow_unsafe_interrupts module option. You'd hit the same problem with the fglrx driver claiming the interrupt even if you had interrupt remapping working.