[vfio-users] GVTg and hybrid graphics (Optimus) setups in Windows guests

Sat Apr 21 22:29:46 UTC 2018

Over the last few months, myself and a few others have been working to
try and get VM assignment of laptop Nvidia dGPUs working with KVM. This
ended up involving a (very hacky) OVMF patch [1] to load the dGPU's
VBIOS via the ACPI _ROM method; this works for now, but if we're able to
get things working properly, it could be cleaned up into something a bit
more maintainable.

Things generally work without much struggle with Linux guests; the
Nvidia GPU can be used to run guest applications both via the Bumblebee
project's optirun/primusrun, and via DRI PRIME. As far as Windows guests
are concerned, we've made significant progress, to the point where the
Nvidia card initialises correctly and applications that explicitly make
use of cross-adapter resources [2] can successfully use the dGPU and
render at near native performance levels.

Unfortunately, under Windows, the vast majority of applications seem to
require a valid [3] hybrid graphics setup for the guest's Direct3D
runtime to select the dGPU to render on. This has lead us down the path
to attempting to get GVT-g working in the hopes that a VM with both a
GVT-g iGPU and a Nvidia dGPU assigned would finally result in a working
set up.

After quite a bit of experimentation and and another OVMF patch [4], we
finally got GVT-g display output working with a UEFI + Windows guest.
This resulted in both the GVT-g iGPU and the Nvidia dGPU initialising
correctly, and showing up in Windows Task Manager's performance tab.
Sadly, the majority of applications will only render on the iGPU, again
the only applications making use of the Nvidia dGPU being those that
explicitly look for cross-adapter resources (in my testing, this is
limited to Unreal Engine 4 games).

After some scrutiny of the Microsoft documentation [3]:

> When the system boots, one of the display adapters is marked as the
current POST adapter. If this POST adapter supports Windows Display
Driver Model (WDDM) 1.3 and has an integrated display panel, it's
considered an /integrated hybrid/ adapter.

I suspect we're getting caught up on the "has an integrated display
panel" (identified by a _DXGK_INTEGRATED_DISPLAY_CHILD [5] child in the
iGPU's _DXGK_CHILD_DESCRIPTOR [6] array populated by the
DXGKDDI_QUERY_CHILD_RELATIONS method) part, as the virtual display
emulated by a GVT-g device is a non-embedded DisplayPort display [8][9].

Assuming that this is the sole issue preventing this setup from working,
then we have several options available but I'm not familiar enough with
this area to accurately gauge the viability of each approach:

1. Amend gvt/display.c in the host kernel such that the virtual display
used by the Intel VGPU is identified as an integrated display panel.

2. See if UPT assignment of the iGPU with x-igd-opregion=on results in
the host's eDP panel being exposed to the guest. I have not been able to
get UPT with x-igd-opregion to show any displays in the guest however,
even though the Windows Intel driver appears to be working properly.

3. See if legacy iGPU assignment results in the host's eDP panel being
exposed to the guest. This would most likely necessitate the use of a
SeaBIOS VM rather than OVMF, as no UEFI-compatible Intel VBIOS exists to
my knowledge. This would also require us to patch SeaBIOS for ACPI ROM
loading for the Nvidia dGPU as in [1]. I've also not been able to get
legacy assignment working on my machine at all - starting the VM
immediately hangs the host (network connectivity is lost and SSH
sessions to the host are dropped)

4. Do this without GVT entirely; upgrade qxl-wddm-dod [10] to WDDM 1.3
(it's currently 1.2 [11]) and amend it such that the emulated monitor is
exposed as an integrated display panel. This is probably the cleanest
approach, if it's technically feasible.

At this point, I'm not really sure how to proceed any further with this
effort. I'm posting this to both the igvt-g and vfio-users lists in the
hopes that the wider audience will lead to someone more knowledgeable on
all this than myself being able to provide some direction/advice as to
how to best advance things.

Cheers,
Jack

[1] https://github.com/jscinoz/optimus-vfio-docs/issues/2
[2]
https://docs.microsoft.com/en-us/windows-hardware/drivers/display/rendering-on-a-discrete-gpu-using-cross-adapter-resources
[3]
https://docs.microsoft.com/en-us/windows-hardware/drivers/display/validating-a-hybrid-system-configuration
[4] https://bugzilla.tianocore.org/show_bug.cgi?id=935
[5]
https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/dispmprt/ns-dispmprt-_dxgk_integrated_display_child
[6]
https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/dispmprt/ns-dispmprt-_dxgk_child_descriptor
[7]
https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/dispmprt/nc-dispmprt-dxgkddi_query_child_relations
[8]
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/gpu/drm/i915/gvt/display.c#n284
[9]
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/gpu/drm/i915/gvt/display.c#n86
[10] https://gitlab.com/spice/qxl-wddm-dod
[11]
https://gitlab.com/spice/qxl-wddm-dod/blob/master/qxldod/QxlDod.cpp#L523

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20180422/72c5a0e8/attachment.sig>