[vfio-users] [iGVT-g] [PATCH v3 00/11] igd passthrough chipset tweaks

[Kay, Allen M]

> -----Original Message-----
> From: Alex Williamson [mailto:alex.williamson at redhat.com]
> Sent: Sunday, January 31, 2016 9:42 AM
> To: Kay, Allen M; Gerd Hoffmann; David Woodhouse
> Cc: igvt-g at ml01.01.org; xen-devel at lists.xensource.com; Eduardo Habkost;
> Stefano Stabellini; qemu-devel at nongnu.org; Cao jin; vfio-
> users at redhat.com
> Subject: Re: [iGVT-g] [vfio-users] [PATCH v3 00/11] igd passthrough chipset
> tweaks
> 
> On Sat, 2016-01-30 at 01:18 +0000, Kay, Allen M wrote:
> >
> > > -----Original Message-----
> > > From: iGVT-g [mailto:igvt-g-bounces at lists.01.org] On Behalf Of Alex
> > > Williamson
> > > Sent: Friday, January 29, 2016 10:00 AM
> > > To: Gerd Hoffmann
> > > Cc: igvt-g at ml01.01.org; xen-devel at lists.xensource.com; Eduardo
> > > Habkost; Stefano Stabellini; qemu-devel at nongnu.org; Cao jin; vfio-
> > > users at redhat.com
> > > Subject: Re: [iGVT-g] [vfio-users] [PATCH v3 00/11] igd passthrough
> > > chipset tweaks
> > >
> > > Do guest drivers depend on IGD appearing at 00:02.0?  I'm currently
> > > testing for any Intel VGA device, but I wonder if I should only be
> > > enabling anything opregion if it also appears at a specific address.
> > >
> >
> > No.  Both Windows and Linux IGD driver should work at any PCI slot.  We
> have seen 0:5.0 in the guest and the driver works.
> 
> Thanks Allen.  Another question, when I boot a VM with an assigned HD
> P4000 GPU, my console stream with IOMMU faults, like:
> 
> DMAR: DMAR:[DMA Write] Request device [00:02.0] fault addr 9fa30000
> DMAR: DMAR:[DMA Write] Request device [00:02.0] fault addr 9fa30000
> DMAR: DMAR:[DMA Write] Request device [00:02.0] fault addr 9fa30000
> DMAR: DMAR:[DMA Write] Request device [00:02.0] fault addr 9fa30000
> DMAR: DMAR:[DMA Write] Request device [00:02.0] fault addr 9fa30000
> 
> All of these fall within the host RMRR range for the device:
> 
> DMAR: Setting RMRR:
> DMAR: Setting identity map for device 0000:00:02.0 [0x9f800000 - 0xaf9fffff]

Hi Alex,

Do you configure IGD as primary or secondary display in your KVM setup?   If primary, are you running Intel vBIOS as part of guest boot?

On BDW/SKL systems, we have started to configure IGD as secondary and QEMU VGA and primary.  In this setup, we are no longer running vBIOS in the guest which avoids some complications.  vBIOS uses stolen memory for display buffers which requires RMRR mapping.  We have been using similar setup (IGD as secondary) on other hypervisors and have not seen IOMMU faults.

I will setup a KVM configuration on my SKL and see if I can duplicate your problem here.   I will try to call into Don's Thursday meeting to discuss this (I'm on call for jury duty this week).  I will give you a heads up on Wednesday evening.

Allen

> 
> A while back, we excluded devices using RMRRs from participating in IOMMU
> API domains because they may continue to DMA to these reserved regions
> after assignment, possibly corrupting VM memory (c875d2c1b808).  Intel
> later decided this exclusion shouldn't apply to graphics devices
> (18436afdc11a).  Don't the above IOMMU faults reveal that exactly the
> problem we're trying to prevent by general exclusion of RMRR encumbered
> devices from the IOMMU API is actually occuring?  If I were to have VM
> memory within the RMRR address range, I wouldn't be seeing these faults,
> I'd be having the GPU corrupt my VM memory.
> 
> David notes in the latter commit above:
> 
> "We should be able to successfully assign graphics devices to guests too, as
> long as the initial handling of stolen memory is reconfigured appropriately."
> 
> What code is supposed to be doing that reconfiguration when a device is
> assigned?  Clearly we don't have it yet, making assignment of these devices
> very unsafe.  It seems like vfio or IOMMU code  in the kernel needs device
> specific code to clear these settings to make it safe for userspace, then
> perhaps VM BIOS support to reallocate.  Is there any consistency across IGD
> revisions for doing this?  Is there a spec?
> Thanks,
> 
> Alex