[vfio-users] VFIO_MAP_DMA error EINVAL

cprt cprt at protonmail.com
Wed Dec 11 17:08:09 UTC 2019 

Hello Alex,
I tried the suggested changes on kernel 5.4.2 and now it is working perfectly.

Thank you for your detailed answer!


Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday 11 December 2019 16:23, Alex Williamson <alex.williamson at redhat.com> wrote:

> On Wed, 11 Dec 2019 13:17:18 +0000
> cprt cprt at protonmail.com wrote:
>
> > Hello,
> > I am using VFIO with QEMU trying to passthrough my audio device.
> > I successfully did this operation with my previous system, with a 7th generation intel and an older kernel.
> > Now I am using a 10th generation intel and a newer kernel (5.4), and I can no longer make this work.
> > Looking at the QEMU code and errors, I can see that this call is failing:
> > ioctl(container->fd, VFIO_IOMMU_MAP_DMA, &map)
> > with error 22 (EINVAL).
> > This is the qemu log:
> > qemu-system-x86_64: -device vfio-pci,host=00:1f.3: vfio_dma_map(0x7f86fdc7c480, 0xc0000, 0x7ff40000, 0x7f83f20c0000) = -22 (Invalid argument)
> > I have setup my system as follows:
> > $ cat /proc/cmdline
> > BOOT_IMAGE=/vmlinuz-linux root=UUID=5b780644-8382-11e9-a363-1b71c3bf36e9 rw loglevel=3 i915.enable_psr=0 intel_iommu=on iommu=pt i915.enable_gvt=1 kvm.ignore_msrs=1 vfio-pci.ids=8086:0284,8086:02c8,8086:02a3,8086:02a4
> > And this is the information of my PCI layout:
> > $ lspci -s 00:1f
> > 00:1f.0 ISA bridge: Intel Corporation Device 0284
> > 00:1f.3 Audio device: Intel Corporation Device 02c8
> > 00:1f.4 SMBus: Intel Corporation Device 02a3
> > 00:1f.5 Serial bus controller [0c80]: Intel Corporation Comet Lake SPI (flash) Controller
> > $ lspci -s 00:1f -n
> > 00:1f.0 0601: 8086:0284
> > 00:1f.3 0403: 8086:02c8
> > 00:1f.4 0c05: 8086:02a3
> > 00:1f.5 0c80: 8086:02a4
> > I tried (just as an experiment) to bypass the error check in the QEMU code, and the virtualized audio device works as expected for a while, then it stops.
> > Do you know what could generate the problem?
>
> I think this is a result of the reserved region support added in v5.4
> which intends to prevent userspace from mapping ranges it shouldn't.
> On my system I have:
>
> $ lspci -nns 00:1f.
> 00:1f.0 ISA bridge [0601]: Intel Corporation Sunrise Point LPC Controller/eSPI Controller [8086:9d4e] (rev 21)
> 00:1f.2 Memory controller [0580]: Intel Corporation Sunrise Point-LP PMC [8086:9d21] (rev 21)
> 00:1f.3 Audio device [0403]: Intel Corporation Sunrise Point-LP HD Audio [8086:9d71] (rev 21)
> 00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-LP SMBus [8086:9d23] (rev 21)
> 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (4) I219-LM [8086:15d7] (rev 21)
>
> $ readlink -f /sys/bus/pci/devices/0000:00:1f.3/iommu_group
> /sys/kernel/iommu_groups/9
>
> $ find /sys/kernel/iommu_groups/9/devices -type l
> /sys/kernel/iommu_groups/9/devices/0000:00:1f.2
> /sys/kernel/iommu_groups/9/devices/0000:00:1f.0
> /sys/kernel/iommu_groups/9/devices/0000:00:1f.3
> /sys/kernel/iommu_groups/9/devices/0000:00:1f.6
> /sys/kernel/iommu_groups/9/devices/0000:00:1f.4
>
> $ cat /sys/kernel/iommu_groups/9/reserved_regions
> 0x0000000000000000 0x0000000000ffffff direct
> 0x00000000fee00000 0x00000000feefffff msi
>
> This direct range seems to be the trouble, your error indicates the
> problem occurs when QEMU tries to map the GPA range 0xc0000-0x7ff40000,
> which clearly overlaps 0x0-0xffffff. It seems this reserved range
> comes from this code:
>
> drivers/iommu/intel-iommu.c:
> #ifdef CONFIG_INTEL_IOMMU_FLOPPY_WA
> if (dev_is_pci(device)) {
> struct pci_dev *pdev = to_pci_dev(device);
>
> if ((pdev->class >> 8) == PCI_CLASS_BRIDGE_ISA) {
>
>                         reg = iommu_alloc_resv_region(0, 1UL << 24, 0,
>                                                       IOMMU_RESV_DIRECT);
>                         if (reg)
>                                 list_add_tail(&reg->list, head);
>
>                 }
>         }
>
>
> #endif /* CONFIG_INTEL_IOMMU_FLOPPY_WA */
>
> We can see above that we do have an ISA bridge in the same IOMMU group
> as the audio device, so this is effectively fallout from the poor
> grouping of these onboard devices.
>
> This code was introduced in v5.3 via:
>
> commit d850c2ee5fe2259968e3889624ad22ea15cb4a38
> Author: Lu Baolu baolu.lu at linux.intel.com
> Date: Sat May 25 13:41:24 2019 +0800
>
> iommu/vt-d: Expose ISA direct mapping region via iommu_get_resv_regions
>
> To support mapping ISA region via iommu_group_create_direct_mappings,
> make sure its exposed by iommu_get_resv_regions.
>
> Signed-off-by: James Sewart jamessewart at arista.com
>
>     Signed-off-by: Lu Baolu <baolu.lu at linux.intel.com>
>
>     Signed-off-by: Joerg Roedel <jroedel at suse.de>
>
>
> The original Kconfig option this relates to was added long ago in:
>
> commit 49a0429e53f29109cbf1eadd89497286ba81f1ae
> Author: Keshavamurthy, Anil S anil.s.keshavamurthy at intel.com
> Date: Sun Oct 21 16:41:57 2007 -0700
>
> Intel IOMMU: Iommu floppy workaround
>
> This config option (DMAR_FLPY_WA) sets up 1:1 mapping for the floppy device so
> that the floppy device which does not use DMA api's will continue to work.
>
> Once the floppy driver starts using DMA api's this config option can be turn
> off or this patch can be yanked out of kernel at that time.
>
> So AIUI the original floppy workaround created an identity map for the
> range 0-16MB. The reserved region attempts to reflect that reservation
> to userspace, however I believe this is a software imposed reserved
> region for the benefit of legacy kernel drivers and should therefore
> use the relaxable version of the reserved region as introduced in
> adfd37382090 ("iommu: Introduce IOMMU_RESV_DIRECT_RELAXABLE reserved
> memory regions"). vfio is able to allow userspace to ignore relaxable
> direct mapping reserved ranges. I think the fix is then simply:
>
> diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
> index 0c8d81f56a30..6eb0dd7489a1 100644
> --- a/drivers/iommu/intel-iommu.c
> +++ b/drivers/iommu/intel-iommu.c
> @@ -5737,7 +5737,7 @@ static void intel_iommu_get_resv_regions(struct device *device,
>
> if ((pdev->class >> 8) == PCI_CLASS_BRIDGE_ISA) {
>
>     		reg = iommu_alloc_resv_region(0, 1UL << 24, 0,
>
>
> -         				      IOMMU_RESV_DIRECT);
>
>
>
> -         				   IOMMU_RESV_DIRECT_RELAXABLE);
>           	if (reg)
>           		list_add_tail(&reg->list, head);
>
>           }
>
>
>
> If you're able to build a kernel with this change, verification would
> be appreciated. I'll post a patch with stable fixes tags unless there
> are objections to this solution. Thanks,
>
> Alex

More information about the vfio-users mailing list