[vfio-users] Question about integrated GPU passthrough and initialization

[Micah Morton]

Hi Alex,

Thanks for the help on this earlier, I was able to get IGD passthrough
working on my device (In case you're interested, crbug.com/970820 has
further details on the changes we needed to make to the kernel/i915
driver to get things working).

I have a couple follow up questions that hopefully you can get to when
you get a chance:

1) I see in https://patchwork.kernel.org/patch/9115161/ that you wrote
a patch for SeaBIOS to reserve stolen memory for the IGD device. It
seems that it is hard-coded to reserve 8MB of stolen memory. I wanted
to reserve 64MB, but wasn't able to locate the "etc/igd-bdsm-size"
file that should be able to configure this. Where does this file live?
Its easy enough for me to hard-code SeaBIOS to 64MB, but I was curious
if there's a cleaner way to set this.

2) Even when SeaBIOS reserves a region for stolen memory, the VM
kernel has a hard time realizing the region is there and available. So
far I just hard-coded the kernel/i915 driver since I know the
address/range of the memory region that SeaBIOS will reserve for
stolen memory in my case. It requires some hard-coding both in the
kernel (https://elixir.bootlin.com/linux/v4.14.114/source/arch/x86/kernel/early-quirks.c#L436)
and in the driver
(https://elixir.bootlin.com/linux/v4.14.114/source/drivers/gpu/drm/i915/i915_gem_stolen.c#L404).
Are you aware of any discussions around this problem? I wanted to see
if it has already been discussed before looking at proposing some kind
of patch to the kernel/driver.

Thanks!
Micah

On Tue, Jun 4, 2019 at 1:44 PM Micah Morton <mortonm at chromium.org> wrote:
>
> Ok thanks, I figured out my issue. Turns out SeaBIOS in the VM tries
> to map from 0x100000-0x8000000 as "System RAM" (which holds the kernel
> code/data/bss sections). On my device the sbios usually only maps
> 0x100000-0x7a988fff for this purpose. So when the i915 driver tries to
> reserve 0x7c000000-0x7fffffff for the "Graphics Stolen Memory" this
> normally succeeds on my device but fails in the VM since the kernel
> sees something already mapping this area of memory. This can be fixed
> by patching SeaBIOS to only reserve up through 0x7bffffff for the
> "System RAM" for the kernel. Somehow the 4.14 kernel is smart enough
> to remap 0x7c000000-0x7fffffff for the Graphics Stolen Memory even if
> it has been claimed by SeaBIOS for System RAM (so this problem doesn't
> show up when running 4.14 in the VM), but 4.4 is not. I'm trying to
> get the 4.4 kernel working in the VM since I think those drivers have
> been more thoroughly tested for my hardware.
>
> On Mon, Jun 3, 2019 at 3:51 PM Alex Williamson
> <alex.williamson at redhat.com> wrote:
> >
> > On Mon, 3 Jun 2019 14:38:49 -0700
> > Micah Morton <mortonm at chromium.org> wrote:
> >
> > > Hi Alex,
> > >
> > > Could you remind me whether there is a minimum recommended kernel
> > > version to be running in the VM guest when doing GPU passthrough?
> > >
> > > I'm fine running 4.14 in the host, but was looking to see if I could
> > > run 4.4 in the guest and couldn't remember if it is advised to use a
> > > newer kernel in the guest or if there is any reason to have the guest
> > > kernel match the host kernel version?
> >
> > I don't recall any specific guest version dependency.  I've got an old
> > 4.9 guest around and that's about the only direct IGD assignment VM I
> > test on a semi-regular basis.  I do keep that host relatively updated,
> > so it's currently a 5.0/4.9 combination.  No reason to keep them in sync
> > that I know of.  Thanks,
> >
> > Alex