[vfio-users] Multi-gpu device passthrough, HD 6990

[Doug Johnson]

Hi All,

I have an HD 6990 card which is a dual gpu card from 2011 (pre GCN) that I am trying to pass through with vfio.

I successfully passed both GPUs to a VM with QEMU, but seabios gets stuck on an option ROM (which I believe is attached to the second GPU). I can see the card initialize, though, so I know that works. If I specify an empty ROM file to the second GPU, the BIOS boots and I can get into windows. I can install drivers, but it appears that they only use one GPU (so, I cannot enable crossfile between them). I also downloaded a copy of the ROM file for my model (Gigabyte) from techpowerup.com and get the same behavior of getting stuck if I specify the file on both GPUs. The AMD drivers also, notably, don't seem to recognize the card properly saying the generic name of "HD 6900 Series," which I believe isn't the correct behavior, although I haven't tried it in a real windows machine to verify yet.

I suspect the issue may have to do with a PLX bridge chip that shows up in lspci (which is part of the card) but that isn't directly passed to the VM, and the GPU bios hangs trying to do something with that, but I am not entirely sure. 

Attached is the output of my lspci -knn as well as my VM spinup script and the output of seabios debug... Any other relevant system details can be provided. 

I am not sure where to go from here... Perhaps emulating the PCI topology of the card somehow? Maybe even there is a way to pass the PLX bridge chip? 

Thanks for your help in advance

-- 
Doug Johnson <dougvj at dougvj.net>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lspci.output
Type: application/octet-stream
Size: 7897 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20180102/14ab71a0/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: start_win_vm.sh
Type: application/x-sh
Size: 895 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20180102/14ab71a0/attachment.sh>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: seabios.output
Type: application/octet-stream
Size: 5005 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20180102/14ab71a0/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20180102/14ab71a0/attachment.sig>