[vfio-users] 1 GB hugepages cause host crash on guest shutdown with some GPUs

[Dan Ziemba]

On Tue, 2015-12-01 at 02:48 -0500, Dan Ziemba wrote:
> I'm using vfio-pci module, set up with this line in a modprobe.d
> file.
> In Arch, that gets copied into the initramfs so it applies before the
> radeon module loads.
> 
> options vfio-pci disable_vga
> ids=1002:67b0,1002:aac0,1033:0194,8086:8d62
> 
> 
> [84786.643384] Call Trace:
> [84786.643391]  [<ffffffff813ff486>] ? __rpm_callback+0x36/0x90
> [84786.643404]  [<ffffffff81400391>] rpm_idle+0x231/0x2a0
> [84786.643415]  [<ffffffff81400453>] __pm_runtime_idle+0x53/0x70
> [84786.643430]  [<ffffffff81312fe8>] pci_device_remove+0x78/0xc0
> [84786.643444]  [<ffffffff813f5247>]
> __device_release_driver+0x87/0x120
> [84786.643458]  [<ffffffff813f5303>] device_release_driver+0x23/0x30
> [84786.643471]  [<ffffffff813f4105>] unbind_store+0x115/0x160
> [84786.643483]  [<ffffffff813f31e5>] drv_attr_store+0x25/0x40

I think I might have just figured out the issue.  My modprobe option
for vfio-pci is wrong for the sound card part of the GPU.  Should be
1002:aac8, not 1002:aac0.  This led to that device not being bound to
vfio-pci, so libvirt would do it automatically when starting/stopping
the vm.  Apparently unbinding the sound card part of the GPU after use
does not work well.  After fixing the option and rebooting everything,
VM shutdown worked after starting it up and running a benchmark for a
few minutes.  So far so good, hopefully this problem is all user error.

Dan