[vfio-users] (good) working GPU for passthrough?

Tobias Geiger tobias.geiger at vido.info
Wed Jan 30 16:04:10 UTC 2019


Hello!

thanks for all your answers!
I re-tried it with a VEGA64 ... but i seem to miss something, as i still 
get the 100%CPU-and-no-vm-startup picture, if i NOT suspend/resume my 
host....
So it seems - i still suffer from the reset bug, despite the VM being 
100% UEFI and Q35 now.

Let me show you my VM: https://pastebin.com/mzhaK7RW

Do i have to re-assamble the whole tree of the VEGA64 PCI Complex, maybe?


-[0000:00]-+-00.0  Intel Corporation Skylake Host Bridge/DRAM Registers
            +-01.0-[01]--
            +-01.1-[02-04]----00.0-[03-04]----00.0-[04]--+-00.0 Advanced 
Micro Devices, Inc. [AMD/ATI] Vega 10 XL/XT [Radeon RX Vega 56/64]
            |                                            \-00.1 Advanced 
Micro Devices, Inc. [AMD/ATI] Device aaf8

right now i just passthrough 4.0 (the GPU) and 4.1 (the HDMI Audio) - 
NOT the PCI Bridges before (2:00 and 3:00);

Do you see an obvious error, regarding my config?
Thanks very much for your input!

Greetings
Tobias


Am 18.01.19 um 21:31 schrieb Paige Thompson:
> If you have trouble getting an AMD card to work with Q35 try using PCIe upstream and downstream ports instead of pcie-root-port. You can use my config as an example however you only need to use one pcie-upstream-port (having two in my config is just a mistake I made) https://gitlab.com/snippets/1788426#note_127191397
>
> you can find information about upstream and downstream pcie ports in this page: https://libvirt.org/formatdomain.html also all of the properties are explained.
>
> ________________________________________
> From: vfio-users-bounces at redhat.com <vfio-users-bounces at redhat.com> on behalf of Bronek Kozicki <brok at spamcop.net>
> Sent: Friday, January 18, 2019 12:20:43 PM
> To: Kash Pande; vfio-users at redhat.com
> Subject: Re: [vfio-users] (good) working GPU for passthrough?
>
> On Fri, 18 Jan 2019, at 1:39 PM, Bronek Kozicki wrote:
>> On Thu, 17 Jan 2019, at 10:39 PM, Kash Pande wrote:
>>> On 2019-01-17 2:03 p.m., Tobias Geiger wrote:
>>>> I tried a VEGA 64 - i was able to live with the ACS patch needed here
>>>> (Z170 Chipset... not needed with the old HD7800, but whatever...) -
>>>> but i couldn't stand the reset/FLR/Bug which forces you at least to
>>>> suspend/resume the host when you want to reboot only the guest...
>>>
>>> AIUI, this does not happen using the Q35 machine type with proper PCIe
>>> root complex heirarchy.
>>>
>>
>> I do not have great experience with "Q35 machine type with proper PCIe
>> root complex hierarchy". Selection of the chipset type Q35 in the virt-
>> manager upon creation does not seem to work with UEFI type of firmware
>> (as recommended for vfio passthrough). Or perhaps I should have
>> downloaded a more recent Tianocore UEFI firmware for my virtual
>> machines?
>
> So I did some more experiments and learned what I suppose everyone else knew already: if switching to SCSI for CDROM, make sure to change the type of the controller to VirtIO from the default LSI.
>
> Also, the host machine must have EFI firmware for guest to boot from. For ArchLinux the appropriate package is ovmf (or ovmf-git from AUR, for adventurous). Setting nvram in /etc/libvirt/qemu.conf must point to the EFI binaries installed by this package. Other distributions also have this package https://pkgs.org/download/ovmf , but that's not news.
>
> Initially, my attempts were failing because OVMF EFI is unable to boot from LSI SCSI controller. This is obviously unrelated to GPU passthrough, except for me changing the incorrect CDROM type from IDE to SCSI (I would not have that problem if I changed to SATA).
>
> Comparing the .xml it is obvious that virt-manager actually constructs the PCI topology with PCIe root for q35, here is appropriate diff excerpt from two nearly identical machines, one created as i440fx and another as q35 (with the above extra steps).
>
> -    <controller type='pci' index='0' model='pci-root'/>
> +    <controller type='pci' index='0' model='pcie-root'/>
> +    <controller type='pci' index='1' model='pcie-root-port'>
> +      <model name='pcie-root-port'/>
> +      <target chassis='1' port='0x10'/>
> +      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
> +    </controller>
> +    <controller type='pci' index='2' model='pcie-root-port'>
> +      <model name='pcie-root-port'/>
> +      <target chassis='2' port='0x11'/>
> +      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
> +    </controller>
> +    <controller type='pci' index='3' model='pcie-to-pci-bridge'>
> +      <model name='pcie-pci-bridge'/>
> +      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
> +    </controller>
> +    <controller type='pci' index='4' model='pcie-root-port'>
> +      <model name='pcie-root-port'/>
> +      <target chassis='4' port='0x12'/>
> +      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
> +    </controller>
> +    <controller type='pci' index='5' model='pcie-root-port'>
> +      <model name='pcie-root-port'/>
> +      <target chassis='5' port='0x13'/>
> +      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
> +    </controller>
> +    <controller type='pci' index='6' model='pcie-root-port'>
> +      <model name='pcie-root-port'/>
> +      <target chassis='6' port='0x14'/>
> +      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
> +     </controller>
>
> This is starting to make sense, going to try passthrough with q35 and UEFI now.
>
>
> B.
>
> _______________________________________________
> vfio-users mailing list
> vfio-users at redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
> _______________________________________________
> vfio-users mailing list
> vfio-users at redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users





More information about the vfio-users mailing list