[vfio-users] empty iommu_groups?

[Edmund F. Nadolski]

Hi,

I'm a noob to VFIO so hopefully this is not to lame a question.

I'm looking to set up a Linux guest VM with a direct-assigned nvme ssd, 
that I can control by a usermode driver with VFIO.  I enable nested 
virtualization in KVM and set up the iommu parameters on the command line:

Host:
# dmesg | grep Command
[    0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/vmlinuz-5.7.0_0611+ 
root=UUID=<blah> ro resume=UUID=<blah> console=tty0 console=ttyS4,115200 
intel_iommu=on iommu=pt
# modprobe kvm_intel nested=1
# ls /sys/kernel/iommu_groups
0  1  10  11  12  13  14  15  16  17  18  19  2  3  4  5  6  7  8  9


The direct assignment works and I can see the drive in the guest, but 
vfio does not create any iommu groups:

Guest:
# lspci | grep Non
07:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe 
SSD Controller SM981/PM981/PM983
# dmesg | grep -i -e DMAR -e IOMMU
[    0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/vmlinuz-5.7.0 
root=/dev/mapper/fedora_localhost--live-root ro 
resume=/dev/mapper/fedora_localhost--live-swap 
rd.lvm.lv=fedora_localhost-live/root 
rd.lvm.lv=fedora_localhost-live/swap intel_iommu=on iommu=pt
[    0.064051] Kernel command line: 
BOOT_IMAGE=(hd0,msdos1)/vmlinuz-5.7.0 
root=/dev/mapper/fedora_localhost--live-root ro 
resume=/dev/mapper/fedora_localhost--live-swap 
rd.lvm.lv=fedora_localhost-live/root 
rd.lvm.lv=fedora_localhost-live/swap intel_iommu=on iommu=pt
[    0.064102] DMAR: IOMMU enabled
[    0.383193] iommu: Default domain type: Passthrough (set via kernel 
command line)
[    1.467722]     intel_iommu=on
# ls -la /sys/kernel/iommu_groups/
total 0
drwxr-xr-x.  2 root root 0 Jun 15 15:16 .
drwxr-xr-x. 15 root root 0 Jun 15 15:12 ..
#

Clearly I'm missing something in my setup/config, but I'm not sure that 
that could be.  Can anyone please advise?

TIA,
Ed