[vfio-users] vfio not working with vanilla kernel 5.4.22

[Alex Williamson]

On Mon, 24 Feb 2020 10:40:39 +0000
"Bronek Kozicki" <brok at spamcop.net> wrote:

> Heads up to anyone running the latest vanilla kernels - after upgrade
> from 5.4.21 to 5.4.22 one of my VMs lost access to a vfio1
> passed-through GPU. This was restored when I downgraded to 5.4.21 so
> the problem seems related to some patch in version 5.4.22
> 
> Also, when starting the VM, I noticed the hypervisor log flooded with
> messages "BAR 3: can't reserve" like:
> 
> Feb 24 09:49:38 gdansk.lan.incorrekt.net kernel: vfio-pci
> 0000:03:00.0: vfio_ecap_init: hiding ecap 0x1e at 0x258 Feb 24 09:49:38
> gdansk.lan.incorrekt.net kernel: vfio-pci 0000:03:00.0:
> vfio_ecap_init: hiding ecap 0x19 at 0x900 Feb 24 09:49:38
> gdansk.lan.incorrekt.net kernel: vfio-pci 0000:03:00.0: BAR 3: can't
> reserve [mem 0xc0000000-0xc1ffffff 64bit pref] Feb 24 09:49:38
> gdansk.lan.incorrekt.net kernel: vfio-pci 0000:03:00.0: No more image
> in the PCI ROM Feb 24 09:51:43 gdansk.lan.incorrekt.net kernel:
> vfio-pci 0000:03:00.0: BAR 3: can't reserve [mem
> 0xc0000000-0xc1ffffff 64bit pref] Feb 24 09:51:43
> gdansk.lan.incorrekt.net kernel: vfio-pci 0000:03:00.0: BAR 3: can't
> reserve [mem 0xc0000000-0xc1ffffff 64bit pref] Feb 24 09:51:43
> gdansk.lan.incorrekt.net kernel: vfio-pci 0000:03:00.0: BAR 3: can't
> reserve [mem 0xc0000000-0xc1ffffff 64bit pref] Feb 24 09:51:43
> gdansk.lan.incorrekt.net kernel: vfio-pci 0000:03:00.0: BAR 3: can't
> reserve [mem 0xc0000000-0xc1ffffff 64bit pref] Feb 24 09:51:43
> gdansk.lan.incorrekt.net kernel: vfio-pci 0000:03:00.0: BAR 3: can't
> reserve [mem 0xc0000000-0xc1ffffff 64bit pref]
> 
> journalctl -b-2 | grep "vfio-pci 0000:03:00.0: BAR 3: can't reserve"
> | wc -l 2609
> 
> Finally, when shutting down the VM I observed kernel panic on the
> hypervisor:
> 
> [  873.831301] Kernel panic - not syncing: Timeout: Not all CPUs
> entered broadcast exception handler [  874.874008] Shutting down cpus
> with NMI [  874.888189] Kernel Offset: 0x0 from 0xffffffff81000000
> (relocation range: 0xffffffff80000000-0xffffffffbfffffff) [
> 875.074319] Rebooting in 30 seconds..

Tried v5.4.22, not getting anything similar.  Potentially there's a
driver activated in this kernel that wasn't previously on your system
and it's attached itself to part of your device.  Look in /proc/iomem
to see what it might be and disable it.  Thanks,

Alex