[vfio-users] VFIO_MAP_DMA error EINVAL
Alex Williamson
alex.williamson at redhat.com
Wed Dec 11 15:23:04 UTC 2019
On Wed, 11 Dec 2019 13:17:18 +0000
cprt <cprt at protonmail.com> wrote:
> Hello,
> I am using VFIO with QEMU trying to passthrough my audio device.
>
> I successfully did this operation with my previous system, with a 7th generation intel and an older kernel.
> Now I am using a 10th generation intel and a newer kernel (5.4), and I can no longer make this work.
>
> Looking at the QEMU code and errors, I can see that this call is failing:
> ioctl(container->fd, VFIO_IOMMU_MAP_DMA, &map)
> with error 22 (EINVAL).
>
> This is the qemu log:
> qemu-system-x86_64: -device vfio-pci,host=00:1f.3: vfio_dma_map(0x7f86fdc7c480, 0xc0000, 0x7ff40000, 0x7f83f20c0000) = -22 (Invalid argument)
>
> I have setup my system as follows:
> $ cat /proc/cmdline
> BOOT_IMAGE=/vmlinuz-linux root=UUID=5b780644-8382-11e9-a363-1b71c3bf36e9 rw loglevel=3 i915.enable_psr=0 intel_iommu=on iommu=pt i915.enable_gvt=1 kvm.ignore_msrs=1 vfio-pci.ids=8086:0284,8086:02c8,8086:02a3,8086:02a4
>
> And this is the information of my PCI layout:
> $ lspci -s 00:1f
> 00:1f.0 ISA bridge: Intel Corporation Device 0284
> 00:1f.3 Audio device: Intel Corporation Device 02c8
> 00:1f.4 SMBus: Intel Corporation Device 02a3
> 00:1f.5 Serial bus controller [0c80]: Intel Corporation Comet Lake SPI (flash) Controller
> $ lspci -s 00:1f -n
> 00:1f.0 0601: 8086:0284
> 00:1f.3 0403: 8086:02c8
> 00:1f.4 0c05: 8086:02a3
> 00:1f.5 0c80: 8086:02a4
>
> I tried (just as an experiment) to bypass the error check in the QEMU code, and the virtualized audio device works as expected for a while, then it stops.
>
> Do you know what could generate the problem?
I think this is a result of the reserved region support added in v5.4
which intends to prevent userspace from mapping ranges it shouldn't.
On my system I have:
$ lspci -nns 00:1f.
00:1f.0 ISA bridge [0601]: Intel Corporation Sunrise Point LPC Controller/eSPI Controller [8086:9d4e] (rev 21)
00:1f.2 Memory controller [0580]: Intel Corporation Sunrise Point-LP PMC [8086:9d21] (rev 21)
00:1f.3 Audio device [0403]: Intel Corporation Sunrise Point-LP HD Audio [8086:9d71] (rev 21)
00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-LP SMBus [8086:9d23] (rev 21)
00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (4) I219-LM [8086:15d7] (rev 21)
$ readlink -f /sys/bus/pci/devices/0000:00:1f.3/iommu_group
/sys/kernel/iommu_groups/9
$ find /sys/kernel/iommu_groups/9/devices -type l
/sys/kernel/iommu_groups/9/devices/0000:00:1f.2
/sys/kernel/iommu_groups/9/devices/0000:00:1f.0
/sys/kernel/iommu_groups/9/devices/0000:00:1f.3
/sys/kernel/iommu_groups/9/devices/0000:00:1f.6
/sys/kernel/iommu_groups/9/devices/0000:00:1f.4
$ cat /sys/kernel/iommu_groups/9/reserved_regions
0x0000000000000000 0x0000000000ffffff direct
0x00000000fee00000 0x00000000feefffff msi
This direct range seems to be the trouble, your error indicates the
problem occurs when QEMU tries to map the GPA range 0xc0000-0x7ff40000,
which clearly overlaps 0x0-0xffffff. It seems this reserved range
comes from this code:
drivers/iommu/intel-iommu.c:
#ifdef CONFIG_INTEL_IOMMU_FLOPPY_WA
if (dev_is_pci(device)) {
struct pci_dev *pdev = to_pci_dev(device);
if ((pdev->class >> 8) == PCI_CLASS_BRIDGE_ISA) {
reg = iommu_alloc_resv_region(0, 1UL << 24, 0,
IOMMU_RESV_DIRECT);
if (reg)
list_add_tail(®->list, head);
}
}
#endif /* CONFIG_INTEL_IOMMU_FLOPPY_WA */
We can see above that we do have an ISA bridge in the same IOMMU group
as the audio device, so this is effectively fallout from the poor
grouping of these onboard devices.
This code was introduced in v5.3 via:
commit d850c2ee5fe2259968e3889624ad22ea15cb4a38
Author: Lu Baolu <baolu.lu at linux.intel.com>
Date: Sat May 25 13:41:24 2019 +0800
iommu/vt-d: Expose ISA direct mapping region via iommu_get_resv_regions
To support mapping ISA region via iommu_group_create_direct_mappings,
make sure its exposed by iommu_get_resv_regions.
Signed-off-by: James Sewart <jamessewart at arista.com>
Signed-off-by: Lu Baolu <baolu.lu at linux.intel.com>
Signed-off-by: Joerg Roedel <jroedel at suse.de>
The original Kconfig option this relates to was added long ago in:
commit 49a0429e53f29109cbf1eadd89497286ba81f1ae
Author: Keshavamurthy, Anil S <anil.s.keshavamurthy at intel.com>
Date: Sun Oct 21 16:41:57 2007 -0700
Intel IOMMU: Iommu floppy workaround
This config option (DMAR_FLPY_WA) sets up 1:1 mapping for the floppy device so
that the floppy device which does not use DMA api's will continue to work.
Once the floppy driver starts using DMA api's this config option can be turn
off or this patch can be yanked out of kernel at that time.
So AIUI the original floppy workaround created an identity map for the
range 0-16MB. The reserved region attempts to reflect that reservation
to userspace, however I believe this is a software imposed reserved
region for the benefit of legacy kernel drivers and should therefore
use the relaxable version of the reserved region as introduced in
adfd37382090 ("iommu: Introduce IOMMU_RESV_DIRECT_RELAXABLE reserved
memory regions"). vfio is able to allow userspace to ignore relaxable
direct mapping reserved ranges. I think the fix is then simply:
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 0c8d81f56a30..6eb0dd7489a1 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -5737,7 +5737,7 @@ static void intel_iommu_get_resv_regions(struct device *device,
if ((pdev->class >> 8) == PCI_CLASS_BRIDGE_ISA) {
reg = iommu_alloc_resv_region(0, 1UL << 24, 0,
- IOMMU_RESV_DIRECT);
+ IOMMU_RESV_DIRECT_RELAXABLE);
if (reg)
list_add_tail(®->list, head);
}
If you're able to build a kernel with this change, verification would
be appreciated. I'll post a patch with stable fixes tags unless there
are objections to this solution. Thanks,
Alex
More information about the vfio-users
mailing list