[vfio-users] Problems passing through Creative X-Fi PCIe sound card

[Alex Williamson]

On Thu, 13 Feb 2020 13:41:44 +1100
Michael Slade <mslade at epic-code.com.au> wrote:

> Hi everyone,
> 
> I'll attempt to start with enough info to describe my situation without 
> pasting a complete `lspci -vvvxxx` output etc.
> 
> My special sound card doesn't want to work when passed through to a guest.
> 
> The card is a Creative "X-Fi Titanium Fatal1ty Pro" (1102:000b), and the 
> mobo is an Asus "TUF Gaming X570 Plus (Wi-Fi)".
> 
> I have already had much success passing through multiple other PCIe 
> devices, including the primary GPU with its HDMI audio, 2 USB 
> controllers and a SATA controller, so I think I have the general process 
> down.  The VM's base is pc-i440fx-3.1, running under libvirt.
> 
> So when I attempt to use the card in the guest, within the first few 
> minutes of running the host goes:
> 
> Feb 13 11:48:01 mickpc-bullseye kernel: [13186.228134] irq 98: nobody 
> cared (try booting with the "irqpoll" option)
> [snip]
> Feb 13 11:48:01 mickpc-bullseye kernel: [13186.228199] handlers:
> Feb 13 11:48:01 mickpc-bullseye kernel: [13186.228202] 
> [<000000009b052715>] vfio_intx_handler [vfio_pci]
> Feb 13 11:48:01 mickpc-bullseye kernel: [13186.228203] Disabling IRQ #98
> 
> Then sound stops working for *some* clients on the guest (including 
> pulseaudio).  The sound goes choppy, as if it's playing each 0.2s of 
> audio 4-5 times.
> 
> I think i tried irqpoll a while back with no luck (From my understanding 
> of what it does, it won't help here anyway)
> 
> I have tried multiple kernel versions on both host and guest (haven't 
> tried 5.5 yet though).
> 
> /proc/interrupts on the host says:
>    98:          0          0          0          0 0          0     
> 400000          0  IR-IO-APIC    7-fasteoi vfio-intx(0000:04:00.0)
> 
> And on the guest:
>   11:          0       7110          0          0 0          0          
> 0          0   IO-APIC  11-fasteoi virtio3, uhci_hcd:usb1, snd_ctxfi
> 
> This is the only device whose interrupt is MSI on the host and not on 
> the guest, and also the only device which is sharing interrupts with 
> other devices on the guest.

VFIO doesn't run in different modes between host and guest, what you
show here is INTx in host and guest.
 
> Can anyone could shed some light on what is actually happening here and 
> how it could be fixed?

I'd guess the device probes OK for DisINTx support, but it doesn't
actually work, the interrupt continues to fire but vfio-pci says "not
my interrupt, my device is masked", when actually it is.  There's a
vfio-pci module option, nointxmask=1 you can use to disable this PCI
2.3 required feature and mask at the IOAPIC instead.  The difficulty is
that vfio needs to be able to get an exclusive interrupt for the
device when using IOAPIC masking, which might mean you need to unbind
anything sharing the interrupt in the host.  The extra bummer is that
it's a global option, so you'll need to do the same for all other
assigned devices.  If it works, we can specifically blacklist the device
in the kernel (drivers/pci/quirks.c) using the quirk_broken_intx_masking
function in the fixup so that the vfio-pci module option is not
required and you'll only need to make sure the audio card has an
exclusive interrupt. TL;DR, the device probes ok for an interrupt
masking feature it doesn't support and interrupts for the card die.
Thanks,

Alex