[vfio-users] vfio passthrough devices behind pcie switcher problem

[Alex Williamson]

On Fri, 10 Mar 2017 10:42:28 +0800
rhett rhett <rhett.kernel at gmail.com> wrote:

> sorry , update infomation right now
> 
> 
> i installed centos7.3 at my 8 gpus machine yesterday, and i made a
> successful passthrough, the vm guest os can use gpu with no problem. so i
> think this is a software problem, i need to patch some patch.
> 
> i also made a test in my 4 gpus machine without any software change, the
> result is success. the 4 gpus are attached at pci root without pcie switch
> , so , i think the software problem have some correlation with pcie switch .
> 
> thank you Alex .
> 
> [root at 64 /data]# lspci|grep NV
> 04:00.0 3D controller: NVIDIA Corporation Device 17fd (rev a1)

Tesla M40

> 05:00.0 3D controller: NVIDIA Corporation Device 1b38 (rev a1)

Tesla P40

> 08:00.0 3D controller: NVIDIA Corporation Device 1b38 (rev a1)
> 09:00.0 3D controller: NVIDIA Corporation Device 1b38 (rev a1)
> 85:00.0 3D controller: NVIDIA Corporation Device 1b38 (rev a1)
> 86:00.0 3D controller: NVIDIA Corporation Device 17fd (rev a1)
> 89:00.0 3D controller: NVIDIA Corporation Device 1b38 (rev a1)
> 8a:00.0 3D controller: NVIDIA Corporation Device 1b38 (rev a1)

Note that you have all of these in 3D mode, you may need to put these
into VGA mode.  NVIDIA provides tools to do this in the host. 

> [root at 64 /data]# lspci -t
> -+-[0000:ff]-+-08.0
>  |           +-08.2
>  |           +-08.3
>  |           +-09.0

[snip... not useful w/o info per devices]
 
> and the xml
> 
> <domain type='kvm'>
>   <name>win</name>
>   <uuid>a2021423-89d8-4a33-aaa5-07102ae7ad4e</uuid>
>   <memory unit='KiB'>8388608</memory>
>   <currentMemory unit='KiB'>8388608</currentMemory>
>   <vcpu placement='static' cpuset='0-8'>8</vcpu>
>   <sysinfo type='smbios'>
>     <system>
>       <entry name='serial'>21aa32e5-8233-40d4-b323-128824f6becf</entry>
>       <entry name='uuid'>a2021423-89d8-4a33-aaa5-07102ae7ad4e</entry>
>     </system>
>   </sysinfo>
>   <os>
>     <type arch='x86_64' machine='pc'>hvm</type>
>     <boot dev='hd'/>
>     <smbios mode='sysinfo'/>
>   </os>
>   <features>
>     <acpi/>
>     <apic/>
>     <pae/>
>     <hap/>
>     <hyperv>
>       <relaxed state='on'/>
>     </hyperv>
>   </features>
>   <cpu>
>     <topology sockets='2' cores='6' threads='2'/>
>   </cpu>
>   <clock offset='localtime'>
>     <timer name='pit' tickpolicy='delay'/>
>     <timer name='rtc' tickpolicy='catchup' track='guest'/>
>     <timer name='hpet' present='no'/>
>   </clock>
>   <on_poweroff>destroy</on_poweroff>
>   <on_reboot>restart</on_reboot>
>   <on_crash>restart</on_crash>
>   <devices>
>     <emulator>/usr/local/bin/qemu-system-x86_64</emulator>

Running custom QEMU, what version is this?