[vfio-users] Hotplug problem

[Patrick O'Callaghan]

On Sun, 2017-11-05 at 08:47 +0100, Hristo Iliev wrote:
> On 05.11.2017 00:53, Patrick O'Callaghan wrote:
> > On Sat, 2017-11-04 at 12:09 +0000, Ferdinand Pöll wrote:
> > > > Is there some special magic to make this work, or is this a known bug
> > > > in QEMU/KVM and I should just forget about it?
> > > 
> > > This sounds like a common problem which occurs when you are using a 
> > > GPU in a VM without Message-Signal-based-Interrupts. Please try this:
> > > * 
> > > https://wiki.lime-technology.com/UnRAID_Manual_6#Enable_MSI_for_Interrupts_to_Fix_HDMI_Audio_Support
> > > * 
> > > https://forums.guru3d.com/threads/windows-line-based-vs-message-signaled-based-interrupts.378044/
> > 
> > Thanks, I took a look and although the PCIe board and GPU are both
> > listed as MSI-capable by lspci on the host, they don't offer that in
> > the Windows registry.
> > 
> 
> The registry keys are often not present and have to be created manually. 
> See Alex's blog entry here:
> http://vfio.blogspot.de/2014/09/vfio-interrupts-and-how-to-coax-windows.html

Thanks. I got that to work with the GPU as an experiment (didn't seem
to make any difference to performance), but I'm still having trouble
with the PCIe card. It seems quite flaky. Before starting the VM 'lspci
-v' sees it:

    04:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 10) (prog-if 30 [XHCI])
            Subsystem: Fresco Logic FL1100 USB 3.0 Host Controller
            Flags: bus master, fast devsel, latency 0, IRQ 18
            Memory at f7800000 (64-bit, non-prefetchable) [size=64K]
            Memory at f7811000 (64-bit, non-prefetchable) [size=4K]
            Memory at f7810000 (64-bit, non-prefetchable) [size=4K]
            Capabilities: [40] Power Management version 3
            Capabilities: [50] MSI: Enable- Count=1/8 Maskable- 64bit+
            Capabilities: [70] Express Endpoint, MSI 00
            Capabilities: [b0] MSI-X: Enable+ Count=8 Masked-
            Capabilities: [100] Advanced Error Reporting
            Kernel driver in use: xhci_hcd

Note that MSI shows as Enable- and MSI-X as Enable+.

Then while the VM is running it doesn't show:

    04:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev ff) (prog-if ff)
            !!! Unknown header type 7f
            Kernel driver in use: vfio-pci

which I guess is normal as the kernel driver has changed. However after
shutting down the VM I then get:

    04:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev ff) (prog-if ff)
            !!! Unknown header type 7f

and have to reboot the host to get it back. I haven't even got to try
the MSI thing yet because sometimes the VM won't even start with it
passed through after this happens, so perhaps the card itself needs
some other kind of reset. I confess ignorance on this.

I'll keep hammering at it ...

poc