[vfio-users] (good) working GPU for passthrough?

[Paige Thompson]

If you have trouble getting an AMD card to work with Q35 try using PCIe upstream and downstream ports instead of pcie-root-port. You can use my config as an example however you only need to use one pcie-upstream-port (having two in my config is just a mistake I made) https://gitlab.com/snippets/1788426#note_127191397

you can find information about upstream and downstream pcie ports in this page: https://libvirt.org/formatdomain.html also all of the properties are explained.

________________________________________
From: vfio-users-bounces at redhat.com <vfio-users-bounces at redhat.com> on behalf of Bronek Kozicki <brok at spamcop.net>
Sent: Friday, January 18, 2019 12:20:43 PM
To: Kash Pande; vfio-users at redhat.com
Subject: Re: [vfio-users] (good) working GPU for passthrough?

On Fri, 18 Jan 2019, at 1:39 PM, Bronek Kozicki wrote:
> On Thu, 17 Jan 2019, at 10:39 PM, Kash Pande wrote:
> > On 2019-01-17 2:03 p.m., Tobias Geiger wrote:
> > > I tried a VEGA 64 - i was able to live with the ACS patch needed here
> > > (Z170 Chipset... not needed with the old HD7800, but whatever...) -
> > > but i couldn't stand the reset/FLR/Bug which forces you at least to
> > > suspend/resume the host when you want to reboot only the guest...
> >
> >
> > AIUI, this does not happen using the Q35 machine type with proper PCIe
> > root complex heirarchy.
> >
>
>
> I do not have great experience with "Q35 machine type with proper PCIe
> root complex hierarchy". Selection of the chipset type Q35 in the virt-
> manager upon creation does not seem to work with UEFI type of firmware
> (as recommended for vfio passthrough). Or perhaps I should have
> downloaded a more recent Tianocore UEFI firmware for my virtual
> machines?


So I did some more experiments and learned what I suppose everyone else knew already: if switching to SCSI for CDROM, make sure to change the type of the controller to VirtIO from the default LSI.

Also, the host machine must have EFI firmware for guest to boot from. For ArchLinux the appropriate package is ovmf (or ovmf-git from AUR, for adventurous). Setting nvram in /etc/libvirt/qemu.conf must point to the EFI binaries installed by this package. Other distributions also have this package https://pkgs.org/download/ovmf , but that's not news.

Initially, my attempts were failing because OVMF EFI is unable to boot from LSI SCSI controller. This is obviously unrelated to GPU passthrough, except for me changing the incorrect CDROM type from IDE to SCSI (I would not have that problem if I changed to SATA).

Comparing the .xml it is obvious that virt-manager actually constructs the PCI topology with PCIe root for q35, here is appropriate diff excerpt from two nearly identical machines, one created as i440fx and another as q35 (with the above extra steps).

-    <controller type='pci' index='0' model='pci-root'/>
+    <controller type='pci' index='0' model='pcie-root'/>
+    <controller type='pci' index='1' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='1' port='0x10'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
+    </controller>
+    <controller type='pci' index='2' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='2' port='0x11'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
+    </controller>
+    <controller type='pci' index='3' model='pcie-to-pci-bridge'>
+      <model name='pcie-pci-bridge'/>
+      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
+    </controller>
+    <controller type='pci' index='4' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='4' port='0x12'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
+    </controller>
+    <controller type='pci' index='5' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='5' port='0x13'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
+    </controller>
+    <controller type='pci' index='6' model='pcie-root-port'>
+      <model name='pcie-root-port'/>
+      <target chassis='6' port='0x14'/>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
+     </controller>

This is starting to make sense, going to try passthrough with q35 and UEFI now.


B.

_______________________________________________
vfio-users mailing list
vfio-users at redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users