[vfio-users] Ryzen Primary GPU passthrough success and woes
Philip Soares
globalgorrilla at fastmail.fm
Wed Apr 5 03:55:41 UTC 2017
On 3 Apr 2017, at 13:21, Graham Neville wrote:
> Cheers for that. Added it to my config and set hugespaces via kernel
> command line.
>
> BOOT_IMAGE=/vmlinuz-linux
> root=UUID=bf69add2-e36f-453a-b92e-a4343ca20d26 rw
> quiet amd_iommu=on vfio-pci.ids=1002:67b1,1002:aac8 video=efifb:off
> isolcpus=0-9
Are those the core and thread siblings on that CPU?
For a 10 core CPU I reserve a few for VMs like:
nohz_full=2-9,12-19 rcu_nocbs=2-9,12-19 isolcpus=2-7,12-17
> kvm_amd.avic=1 hugepages=12188
Does the CPU support 1G pages? With 1G pages I reserve 32G for VMs like:
default_hugepagesz=1G hugepagesz=1G hugepages=32
transparent_hugepage=never
> I also set iothreads, but can't seem to change my disks from sata to
> virtio. Whenever I try I get a windows bluescreen on boot.
>
> I'm not seeing any interrupts for when the GPU is in use:
I don't have a Ryzen CPU so not sure about this.
I think someone just said yesterday on this list, in response to my
question about it, that they had them with Ryzen, if I read it right.
> PIN: 0 0 0 0 0
> 0 0 0 0 0 0
> 0 0 0 0 0 Posted-interrupt
> notification event
> PIW: 0 0 0 0 0
> 0 0 0 0 0 0
> 0 0 0 0 0 Posted-interrupt
> wakeup
> event
>
>
> Here's my current XML:
>
>
>
> <domain type='kvm'>
> <name>Windows10</name>
> <uuid>d45c3b5f-be8a-41e8-a22c-02e91c4c6245</uuid>
> <memory unit='KiB'>16384000</memory>
> <currentMemory unit='KiB'>16384000</currentMemory>
> <memoryBacking>
> <hugepages/>
> </memoryBacking>
> <vcpu placement='static'>8</vcpu>
> <iothreads>4</iothreads>
> <cputune>
> <vcpupin vcpu='0' cpuset='0'/>
> <vcpupin vcpu='1' cpuset='1'/>
> <vcpupin vcpu='2' cpuset='2'/>
> <vcpupin vcpu='3' cpuset='3'/>
> <vcpupin vcpu='4' cpuset='4'/>
> <vcpupin vcpu='5' cpuset='5'/>
> <vcpupin vcpu='6' cpuset='6'/>
> <vcpupin vcpu='7' cpuset='7'/>
> <iothreadpin iothread='1' cpuset='0-1'/>
> <iothreadpin iothread='2' cpuset='2-3'/>
> <iothreadpin iothread='3' cpuset='4-5'/>
> <iothreadpin iothread='4' cpuset='6-7'/>
> </cputune>
> <os>
> <type arch='x86_64' machine='pc-q35-2.6'>hvm</type>
> <loader readonly='yes'
> type='pflash'>/home/virtualguests/windows10/ovmf_code_x64.bin</loader>
> <nvram>/home/virtualguests/windows10/ovmf_vars_x64.bin</nvram>
> <boot dev='hd'/>
> </os>
> <features>
> <acpi/>
> <apic/>
> <pae/>
> <hyperv>
> <relaxed state='on'/>
> <vapic state='on'/>
> <spinlocks state='on' retries='8191'/>
> </hyperv>
> <kvm>
> <hidden state='on'/>
> </kvm>
> <vmport state='off'/>
> </features>
> <cpu mode='host-passthrough'>
> <topology sockets='1' cores='8' threads='1'/>
> </cpu>
> <clock offset='localtime'>
> <timer name='rtc' tickpolicy='catchup'/>
> <timer name='pit' tickpolicy='delay'/>
> <timer name='hpet' present='no'/>
> <timer name='hypervclock' present='yes'/>
> </clock>
> <on_poweroff>destroy</on_poweroff>
> <on_reboot>restart</on_reboot>
> <on_crash>restart</on_crash>
> <pm>
> <suspend-to-mem enabled='no'/>
> <suspend-to-disk enabled='no'/>
> </pm>
> <devices>
> <emulator>/usr/bin/qemu-system-x86_64</emulator>
> <disk type='file' device='disk'>
> <driver name='qemu' type='qcow2' cache='none'/>
> <source
> file='/home/virtualguests/windows10/windows10-c-nas.qcow2'/>
> <target dev='sda' bus='sata'/>
> <address type='drive' controller='0' bus='0' target='0'
> unit='0'/>
> </disk>
> <disk type='file' device='disk'>
> <driver name='qemu' type='qcow2' cache='none'/>
> <source file='/storage/windows10-d.qcow2'/>
> <target dev='sdb' bus='sata'/>
> <address type='drive' controller='0' bus='0' target='0'
> unit='1'/>
> </disk>
> <controller type='usb' index='0' model='ich9-ehci1'>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x1d'
> function='0x7'/>
> </controller>
> <controller type='usb' index='0' model='ich9-uhci1'>
> <master startport='0'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x1d'
> function='0x0' multifunction='on'/>
> </controller>
> <controller type='usb' index='0' model='ich9-uhci2'>
> <master startport='2'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x1d'
> function='0x1'/>
> </controller>
> <controller type='usb' index='0' model='ich9-uhci3'>
> <master startport='4'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x1d'
> function='0x2'/>
> </controller>
> <controller type='sata' index='0'>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f'
> function='0x2'/>
> </controller>
> <controller type='pci' index='0' model='pcie-root'/>
> <controller type='pci' index='1' model='dmi-to-pci-bridge'>
> <model name='i82801b11-bridge'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e'
> function='0x0'/>
> </controller>
> <controller type='pci' index='2' model='pci-bridge'>
> <model name='pci-bridge'/>
> <target chassisNr='2'/>
> <address type='pci' domain='0x0000' bus='0x01' slot='0x00'
> function='0x0'/>
> </controller>
> <controller type='pci' index='3' model='pcie-root-port'>
> <model name='ioh3420'/>
> <target chassis='3' port='0x40'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
> function='0x0'/>
> </controller>
> <controller type='virtio-serial' index='0'>
> <address type='pci' domain='0x0000' bus='0x02' slot='0x03'
> function='0x0'/>
> </controller>
> <serial type='pty'>
> <target port='0'/>
> </serial>
> <console type='pty'>
> <target type='serial' port='0'/>
> </console>
> <input type='tablet' bus='usb'>
> <address type='usb' bus='0' port='1'/>
> </input>
> <input type='mouse' bus='ps2'/>
> <input type='keyboard' bus='ps2'/>
> <interface type='bridge'>
> <mac address='52:54:00:12:34:76'/>
> <source bridge='br0'/>
> <target dev='tap8'/>
> <model type='virtio'/>
> <alias name='virtio'/>
> <rom bar='off'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
> function='0x0'/>
> </interface>
> <hostdev mode='subsystem' type='usb' managed='no'>
> <source>
> <vendor id='0x046d'/>
> <product id='0xc52e'/>
> </source>
> </hostdev>
> <hostdev mode='subsystem' type='usb' managed='no'>
> <source>
> <vendor id='0x04d8'/>
> <product id='0x00df'/>
> </source>
> </hostdev>
> <hostdev mode='subsystem' type='usb' managed='no'>
> <source>
> <vendor id='0x28de'/>
> <product id='0x1142'/>
> </source>
> </hostdev>
> <hostdev mode='subsystem' type='usb' managed='no'>
> <source>
> <vendor id='0x0a12'/>
> <product id='0x0001'/>
> </source>
> </hostdev>
> <hostdev mode='subsystem' type='usb' managed='no'>
> <source>
> <vendor id='0x0fcf'/>
> <product id='0x1009'/>
> </source>
> </hostdev>
> <hostdev mode='subsystem' type='usb' managed='no'>
> <source>
> <vendor id='0x1b1c'/>
> <product id='0x1c0b'/>
> </source>
> </hostdev>
> <hostdev mode='subsystem' type='usb' managed='no'>
> <source>
> <vendor id='0x2433'/>
> <product id='0xb200'/>
> </source>
> </hostdev>
> <sound model='ich6'>
> <address type='pci' domain='0x0000' bus='0x02' slot='0x02'
> function='0x0'/>
> </sound>
> <hostdev mode='subsystem' type='pci' managed='yes'>
> <source>
> <address domain='0x0000' bus='0x09' slot='0x00'
> function='0x0'/>
> </source>
> <rom bar='on' file='/home/virtualguests/windows10/r9290.rom'/>
> <address type='pci' domain='0x0000' bus='0x03' slot='0x00'
> function='0x0' multifunction='on'/>
> </hostdev>
> <hostdev mode='subsystem' type='pci' managed='yes'>
> <source>
> <address domain='0x0000' bus='0x09' slot='0x00'
> function='0x1'/>
> </source>
> <address type='pci' domain='0x0000' bus='0x03' slot='0x00'
> function='0x1'/>
> </hostdev>
> <memballoon model='none'/>
> </devices>
> </domain>
>
> On Mon, Apr 3, 2017 at 5:13 PM, <globalgorrilla at fastmail.fm> wrote:
>
>> <memballoon model='none'/>
>>
>> Might perform better.
>>
>> Also, yes Hugepages might be helpful too.
>>
>> Also, would you mind observing whether interrupts are posted when the
>> GPU
>> is in use in pass-through?
>>
>> watch -d cat /proc/interrupts
>>
>> Look at the bottom for PIN (Posted-interrupt notification event) and
>> PIW
>> (Posted-interrupt wakeup event).
>>
>> Thanks!
>>
>> On 30 Mar 2017, at 15:46, Graham Neville wrote:
>>
>> Finally gotten to the bottom of this, this the help of your XML file
>> I'm
>> able to run with a Q35 setup, I'm also able to just passthrough the
>> CPU
>> features. Thanks.
>>
>> I discovered the issue with the guest crashing was actually due to my
>> graphics card overheating! Noticed that it was hitting 99C and then
>> guest
>> would crash.
>>
>> The setup is pretty sweet now, I just want to change my drives from
>> sata
>> to virtio which I believe will be quicker. Then also investigate
>> HugePages
>>
>> Here's my Kernel params:
>>
>> cat /proc/cmdline
>> BOOT_IMAGE=/vmlinuz-linux
>> root=UUID=bf69add2-e36f-453a-b92e-a4343ca20d26
>> rw quiet amd_iommu=on vfio-pci.ids=1002:67b1,1002:aac8
>> video=efifb:off
>> isolcpus=0-7
>>
>> Here's my final XML file:
>>
>> <domain type='kvm'>
>> <name>Windows10</name>
>> <uuid>d45c3b5f-be8a-41e8-a22c-02e91c4c6245</uuid>
>> <memory unit='KiB'>8388608</memory>
>> <currentMemory unit='KiB'>8388608</currentMemory>
>> <vcpu placement='static'>4</vcpu>
>> <cputune>
>> <vcpupin vcpu='0' cpuset='0'/>
>> <vcpupin vcpu='1' cpuset='1'/>
>> <vcpupin vcpu='2' cpuset='2'/>
>> <vcpupin vcpu='3' cpuset='3'/>
>> </cputune>
>> <os>
>> <type arch='x86_64' machine='pc-q35-2.6'>hvm</type>
>> <loader readonly='yes' type='pflash'>/home/
>> virtualguests/windows10/ovmf_code_x64.bin</loader>
>> <nvram>/home/virtualguests/windows10/ovmf_vars_x64.bin</nvram>
>> <boot dev='hd'/>
>> </os>
>> <features>
>> <acpi/>
>> <apic/>
>> <pae/>
>> <hyperv>
>> <relaxed state='on'/>
>> <vapic state='on'/>
>> <spinlocks state='on' retries='8191'/>
>> </hyperv>
>> <kvm>
>> <hidden state='on'/>
>> </kvm>
>> </features>
>> <cpu mode='host-passthrough'/>
>> <clock offset='localtime'>
>> <timer name='rtc' tickpolicy='catchup'/>
>> <timer name='pit' tickpolicy='delay'/>
>> <timer name='hpet' present='no'/>
>> <timer name='hypervclock' present='yes'/>
>> </clock>
>> <on_poweroff>destroy</on_poweroff>
>> <on_reboot>restart</on_reboot>
>> <on_crash>restart</on_crash>
>> <pm>
>> <suspend-to-mem enabled='no'/>
>> <suspend-to-disk enabled='no'/>
>> </pm>
>> <devices>
>> <emulator>/usr/bin/qemu-system-x86_64</emulator>
>> <disk type='file' device='disk'>
>> <driver name='qemu' type='qcow2'/>
>> <source
>> file='/home/virtualguests/windows10/windows10-c-nas.qcow2'/>
>> <target dev='sda' bus='sata'/>
>> <address type='drive' controller='0' bus='0' target='0'
>> unit='0'/>
>> </disk>
>> <disk type='file' device='disk'>
>> <driver name='qemu' type='qcow2'/>
>> <source file='/storage/windows10-d.qcow2'/>
>> <target dev='sdb' bus='sata'/>
>> <address type='drive' controller='0' bus='0' target='0'
>> unit='1'/>
>> </disk>
>> <controller type='usb' index='0' model='ich9-ehci1'>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x1d'
>> function='0x7'/>
>> </controller>
>> <controller type='usb' index='0' model='ich9-uhci1'>
>> <master startport='0'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x1d'
>> function='0x0' multifunction='on'/>
>> </controller>
>> <controller type='usb' index='0' model='ich9-uhci2'>
>> <master startport='2'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x1d'
>> function='0x1'/>
>> </controller>
>> <controller type='usb' index='0' model='ich9-uhci3'>
>> <master startport='4'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x1d'
>> function='0x2'/>
>> </controller>
>> <controller type='sata' index='0'>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f'
>> function='0x2'/>
>> </controller>
>> <controller type='pci' index='0' model='pcie-root'/>
>> <controller type='pci' index='1' model='dmi-to-pci-bridge'>
>> <model name='i82801b11-bridge'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e'
>> function='0x0'/>
>> </controller>
>> <controller type='pci' index='2' model='pci-bridge'>
>> <model name='pci-bridge'/>
>> <target chassisNr='2'/>
>> <address type='pci' domain='0x0000' bus='0x01' slot='0x00'
>> function='0x0'/>
>> </controller>
>> <controller type='pci' index='3' model='pcie-root-port'>
>> <model name='ioh3420'/>
>> <target chassis='3' port='0x40'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
>> function='0x0'/>
>> </controller>
>> <controller type='virtio-serial' index='0'>
>> <address type='pci' domain='0x0000' bus='0x02' slot='0x03'
>> function='0x0'/>
>> </controller>
>> <serial type='pty'>
>> <target port='0'/>
>> </serial>
>> <console type='pty'>
>> <target type='serial' port='0'/>
>> </console>
>> <input type='tablet' bus='usb'>
>> <address type='usb' bus='0' port='1'/>
>> </input>
>> <input type='mouse' bus='ps2'/>
>> <input type='keyboard' bus='ps2'/>
>> <interface type='bridge'>
>> <mac address='52:54:00:12:34:76'/>
>> <source bridge='br0'/>
>> <target dev='tap8'/>
>> <model type='virtio'/>
>> <alias name='virtio'/>
>> <rom bar='off'/>
>> </interface>
>> <hostdev mode='subsystem' type='usb' managed='no'>
>> <source>
>> <vendor id='0x046d'/>
>> <product id='0xc52e'/>
>> </source>
>> </hostdev>
>> <hostdev mode='subsystem' type='usb' managed='no'>
>> <source>
>> <vendor id='0x04d8'/>
>> <product id='0x00df'/>
>> </source>
>> </hostdev>
>> <hostdev mode='subsystem' type='usb' managed='no'>
>> <source>
>> <vendor id='0x28de'/>
>> <product id='0x1142'/>
>> </source>
>> </hostdev>
>> <hostdev mode='subsystem' type='usb' managed='no'>
>> <source>
>> <vendor id='0x0a12'/>
>> <product id='0x0001'/>
>> </source>
>> </hostdev>
>> <hostdev mode='subsystem' type='usb' managed='no'>
>> <source>
>> <vendor id='0x0fcf'/>
>> <product id='0x1009'/>
>> </source>
>> </hostdev>
>> <hostdev mode='subsystem' type='usb' managed='no'>
>> <source>
>> <vendor id='0x1b1c'/>
>> <product id='0x1c0b'/>
>> </source>
>> </hostdev>
>> <sound model='ich6'>
>> <address type='pci' domain='0x0000' bus='0x02' slot='0x02'
>> function='0x0'/>
>> </sound>
>> <hostdev mode='subsystem' type='pci' managed='yes'>
>> <source>
>> <address domain='0x0000' bus='0x09' slot='0x00'
>> function='0x0'/>
>> </source>
>> <rom bar='on' file='/home/virtualguests/windows10/r9290.rom'/>
>> <address type='pci' domain='0x0000' bus='0x03' slot='0x00'
>> function='0x0' multifunction='on'/>
>> </hostdev>
>> <hostdev mode='subsystem' type='pci' managed='yes'>
>> <source>
>> <address domain='0x0000' bus='0x09' slot='0x00'
>> function='0x1'/>
>> </source>
>> <address type='pci' domain='0x0000' bus='0x03' slot='0x00'
>> function='0x1'/>
>> </hostdev>
>> <memballoon model='virtio'>
>> <address type='pci' domain='0x0000' bus='0x02' slot='0x04'
>> function='0x0'/>
>> </memballoon>
>> </devices>
>> </domain>
>>
>>
>>
>> On Thu, Mar 30, 2017 at 6:04 AM, Graham Neville
>> <grahamneville at gmail.com>
>> wrote:
>>
>>> I am using a ROM file which I dumped myself using GPU-Z, this was
>>> the
>>> only was I could get a display as well. With Q35 I do see TianoCore
>>> and
>>> then the Windows10 logo but the guest crashes straight after that. I
>>> may
>>> have to try and do a fresh install with Q35 to see if I can get
>>> further.
>>> Thanks for the XML, I'll try again tonight and report back.
>>>
>>>
>>> On 29 Mar 2017 11:33 p.m., "Steven Walter" <stevenrwalter at gmail.com>
>>> wrote:
>>>
>>> I didn't have good luck with Windows on BIOS or with Windows on
>>> i440.
>>> I pretty much had to use Q35 + OVMF. You said you had trouble
>>> getting
>>> Windows to boot with Q35 + OVMF, did you at least get the TianoCore
>>> logo to come up on your screen? If not you may need to dump and
>>> specify the ROM file for your video card. For me OVMF would hang
>>> unless I did this. Note that I'm using Q35 + OVMF, Opteron_G3 as
>>> CPU,
>>> specifying a ROM for the GPU, and running kvm_amd.avic=1.
>>>
>>> Here is my full guest XML file:
>>>
>>> <domain type='kvm'>
>>> <name>WindowsTest</name>
>>> <uuid>d45c3b5f-be8a-41e8-a22c-02e91c4c6245</uuid>
>>> <memory unit='KiB'>8388608</memory>
>>> <currentMemory unit='KiB'>8388608</currentMemory>
>>> <vcpu placement='static'>8</vcpu>
>>> <os>
>>> <type arch='x86_64' machine='pc-q35-2.6'>hvm</type>
>>> <loader readonly='yes' type='pflash'>/usr/share/OVMF/
>>> OVMF_CODE.fd</loader>
>>> <nvram>/var/lib/libvirt/qemu/nvram/WindowsTest_VARS.fd</nvram>
>>> <boot dev='hd'/>
>>> </os>
>>> <features>
>>> <acpi/>
>>> <apic/>
>>> <hyperv>
>>> <relaxed state='on'/>
>>> <vapic state='on'/>
>>> <spinlocks state='on' retries='8191'/>
>>> </hyperv>
>>> </features>
>>> <cpu mode='host-passthrough'/>
>>> <clock offset='localtime'>
>>> <timer name='rtc' tickpolicy='catchup'/>
>>> <timer name='pit' tickpolicy='delay'/>
>>> <timer name='hpet' present='no'/>
>>> <timer name='hypervclock' present='yes'/>
>>> </clock>
>>> <on_poweroff>destroy</on_poweroff>
>>> <on_reboot>restart</on_reboot>
>>> <on_crash>restart</on_crash>
>>> <pm>
>>> <suspend-to-mem enabled='no'/>
>>> <suspend-to-disk enabled='no'/>
>>> </pm>
>>> <devices>
>>> <emulator>/usr/bin/kvm-spice</emulator>
>>> <disk type='file' device='disk'>
>>> <driver name='qemu' type='qcow2'/>
>>> <source file='/var/lib/libvirt/images/WindowsTest.qcow2'/>
>>> <target dev='sda' bus='sata'/>
>>> <address type='drive' controller='0' bus='0' target='0'
>>> unit='0'/>
>>> </disk>
>>> <disk type='file' device='cdrom'>
>>> <driver name='qemu' type='raw'/>
>>> <source file='/home/srwalter/Win10_1607_English_x64.iso'/>
>>> <target dev='sdb' bus='sata'/>
>>> <readonly/>
>>> <address type='drive' controller='0' bus='0' target='0'
>>> unit='1'/>
>>> </disk>
>>> <controller type='usb' index='0' model='ich9-ehci1'>
>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x1d'
>>> function='0x7'/>
>>> </controller>
>>> <controller type='usb' index='0' model='ich9-uhci1'>
>>> <master startport='0'/>
>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x1d'
>>> function='0x0' multifunction='on'/>
>>> </controller>
>>> <controller type='usb' index='0' model='ich9-uhci2'>
>>> <master startport='2'/>
>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x1d'
>>> function='0x1'/>
>>> </controller>
>>> <controller type='usb' index='0' model='ich9-uhci3'>
>>> <master startport='4'/>
>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x1d'
>>> function='0x2'/>
>>> </controller>
>>> <controller type='sata' index='0'>
>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f'
>>> function='0x2'/>
>>> </controller>
>>> <controller type='pci' index='0' model='pcie-root'/>
>>> <controller type='pci' index='1' model='dmi-to-pci-bridge'>
>>> <model name='i82801b11-bridge'/>
>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x1e'
>>> function='0x0'/>
>>> </controller>
>>> <controller type='pci' index='2' model='pci-bridge'>
>>> <model name='pci-bridge'/>
>>> <target chassisNr='2'/>
>>> <address type='pci' domain='0x0000' bus='0x01' slot='0x00'
>>> function='0x0'/>
>>> </controller>
>>> <controller type='pci' index='3' model='pcie-root-port'>
>>> <model name='ioh3420'/>
>>> <target chassis='3' port='0x40'/>
>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
>>> function='0x0'/>
>>> </controller>
>>> <controller type='virtio-serial' index='0'>
>>> <address type='pci' domain='0x0000' bus='0x02' slot='0x03'
>>> function='0x0'/>
>>> </controller>
>>> <serial type='pty'>
>>> <target port='0'/>
>>> </serial>
>>> <console type='pty'>
>>> <target type='serial' port='0'/>
>>> </console>
>>> <channel type='spicevmc'>
>>> <target type='virtio' name='com.redhat.spice.0'/>
>>> <address type='virtio-serial' controller='0' bus='0'
>>> port='1'/>
>>> </channel>
>>> <input type='tablet' bus='usb'>
>>> <address type='usb' bus='0' port='1'/>
>>> </input>
>>> <input type='mouse' bus='ps2'/>
>>> <input type='keyboard' bus='ps2'/>
>>> <graphics type='spice' autoport='yes'>
>>> <listen type='address'/>
>>> </graphics>
>>> <sound model='ich6'>
>>> <address type='pci' domain='0x0000' bus='0x02' slot='0x02'
>>> function='0x0'/>
>>> </sound>
>>> <video>
>>> <model type='cirrus' vram='16384' heads='1' primary='yes'/>
>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
>>> function='0x0'/>
>>> </video>
>>> <hostdev mode='subsystem' type='pci' managed='yes'>
>>> <source>
>>> <address domain='0x0000' bus='0x23' slot='0x00'
>>> function='0x0'/>
>>> </source>
>>> <rom bar='on' file='/usr/share/OVMF/Ellesmere.rom'/>
>>> <address type='pci' domain='0x0000' bus='0x03' slot='0x00'
>>> function='0x0' multifunction='on'/>
>>> </hostdev>
>>> <hostdev mode='subsystem' type='pci' managed='yes'>
>>> <source>
>>> <address domain='0x0000' bus='0x23' slot='0x00'
>>> function='0x1'/>
>>> </source>
>>> <address type='pci' domain='0x0000' bus='0x03' slot='0x00'
>>> function='0x1'/>
>>> </hostdev>
>>> <redirdev bus='usb' type='spicevmc'>
>>> <address type='usb' bus='0' port='2'/>
>>> </redirdev>
>>> <redirdev bus='usb' type='spicevmc'>
>>> <address type='usb' bus='0' port='3'/>
>>> </redirdev>
>>> <memballoon model='virtio'>
>>> <address type='pci' domain='0x0000' bus='0x02' slot='0x04'
>>> function='0x0'/>
>>> </memballoon>
>>> </devices>
>>> </domain>
>>>
>>> On Wed, Mar 29, 2017 at 5:44 PM, Graham Neville
>>> <grahamneville at gmail.com>
>>> wrote:
>>>> Thanks for the link. I've tried a number of things but still no
>>>> further
>>> down
>>>> the line.
>>>>
>>>> I've tried the following one-by-one
>>>>
>>>> kvm_amd.avic=1
>>>> Properly isolated CPUs on host so they are exclusive to guest -
>>> isolcpus=0-7
>>>> Changed CPU host-passthrough to Opteron_G5 instead
>>>> Changed CPU to althon
>>>> Changed CPU to qemu64
>>>> kvm-amd.npt=0
>>>> iommu=pt
>>>> Disabled SMT in BIOS
>>>>
>>>> In using Arch Linux with Kernel 4.10.1.
>>>>
>>>> uname -a
>>>> Linux amdr7 4.10.5-1-ARCH #1 SMP PREEMPT Wed Mar 22 14:42:03 CET
>>>> 2017
>>> x86_64
>>>> GNU/Linux
>>>>
>>>> This is my kernel command line now:
>>>>
>>>> BOOT_IMAGE=/vmlinuz-linux
>>>> root=UUID=bf69add2-e36f-453a-b92e-a4343ca20d26
>>> rw
>>>> quiet amd_iommu=on vfio-pci.ids=1002:67b1,1002:aac8 video=efifb:off
>>>> amdgpu.msi=0 kvm_amd.avic=1 isolcpus=0-7 kvm-amd.npt=0 iommu=pt
>>>>
>>>> This is my full libvirt XML file for the VM:
>>>>
>>>>
>>>> <domain type='kvm'>
>>>> <name>windows10</name>
>>>> <uuid>7b222825-fc7d-4a66-a72c-5876063752d5</uuid>
>>>> <memory unit='KiB'>8291456</memory>
>>>> <currentMemory unit='KiB'>8291456</currentMemory>
>>>> <vcpu placement='static'>4</vcpu>
>>>> <cputune>
>>>> <vcpupin vcpu='0' cpuset='0'/>
>>>> <vcpupin vcpu='1' cpuset='1'/>
>>>> <vcpupin vcpu='2' cpuset='2'/>
>>>> <vcpupin vcpu='3' cpuset='3'/>
>>>> </cputune>
>>>> <os>
>>>> <type arch='x86_64' machine='pc-i440fx-2.1'>hvm</type>
>>>> <loader type='pflash'
>>>> readonly='yes'>/home/virtualguests/windows10/OVMF_CODE.fd</loader>
>>>> <nvram>/home/virtualguests/windows10/OVMF_VARS.fd</nvram>
>>>> <boot dev='hd'/>
>>>> </os>
>>>> <features>
>>>> <acpi/>
>>>> <apic/>
>>>> <pae/>
>>>> <hyperv>
>>>> <relaxed state='on'/>
>>>> <vapic state='on'/>
>>>> <spinlocks state='on' retries='8191'/>
>>>> </hyperv>
>>>> <kvm>
>>>> <hidden state='on'/>
>>>> </kvm>
>>>> </features>
>>>> <cpus>
>>>> <arch name='x86'>
>>>> <model name='kvm64'>
>>>> <feature name='apic'/>
>>>> <feature name='clflush'/>
>>>> <feature name='cmov'/>
>>>> <feature name='cx16'/>
>>>> <feature name='cx8'/>
>>>> <feature name='de'/>
>>>> <feature name='fpu'/>
>>>> <feature name='fxsr'/>
>>>> <feature name='lm'/>
>>>> <feature name='mca'/>
>>>> <feature name='mce'/>
>>>> <feature name='mmx'/>
>>>> <feature name='msr'/>
>>>> <feature name='mtrr'/>
>>>> <feature name='nx'/>
>>>> <feature name='pae'/>
>>>> <feature name='pat'/>
>>>> <feature name='pge'/>
>>>> <feature name='pni'/>
>>>> <feature name='pse'/>
>>>> <feature name='pse36'/>
>>>> <feature name='sep'/>
>>>> <feature name='sse'/>
>>>> <feature name='sse2'/>
>>>> <feature name='syscall'/>
>>>> <feature name='tsc'/>
>>>> </model>
>>>> </arch>
>>>> </cpus>
>>>> <clock offset='localtime'>
>>>> <timer name='rtc' tickpolicy='catchup'/>
>>>> <timer name='pit' tickpolicy='delay'/>
>>>> <timer name='hpet' present='no'/>
>>>> <timer name='hypervclock' present='yes'/>
>>>> </clock>
>>>> <on_poweroff>destroy</on_poweroff>
>>>> <on_reboot>restart</on_reboot>
>>>> <on_crash>destroy</on_crash>
>>>> <devices>
>>>> <emulator>/usr/bin/qemu-system-x86_64</emulator>
>>>> <disk type='file' device='disk'>
>>>> <driver name='qemu' type='qcow2' />
>>>> <source file='/home/virtualguests/wind
>>> ows10/windows10-c-nas.qcow2'/>
>>>> <target dev='vdb' bus='virtio'/>
>>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x07'
>>>> function='0x0'/>
>>>> </disk>
>>>> <disk type='file' device='cdrom'>
>>>> <driver name='qemu' type='raw'/>
>>>> <source
>>>> file='/home/virtualguests/windows10/Win10_1607_EnglishIntern
>>> ational_x64.iso'/>
>>>> <target dev='hdc' bus='ide'/>
>>>> <readonly/>
>>>> <address type='drive' controller='0' bus='1' unit='0'/>
>>>> </disk>
>>>> <disk type='file' device='disk'>
>>>> <driver name='qemu' type='qcow2' cache='none'/>
>>>> <source file='/storage/windows10-d.qcow2'/>
>>>> <target dev='vdc' bus='virtio'/>
>>>> </disk>
>>>> <controller type='pci' index='0' model='pci-root' />
>>>> <interface type='bridge'>
>>>> <mac address='52:54:00:12:34:76'/>
>>>> <source bridge='br0'/>
>>>> <target dev='tap8'/>
>>>> <model type='virtio'/>
>>>> <alias name='virtio'/>
>>>> <rom bar='off'/>
>>>> </interface>
>>>> <input type='mouse' bus='ps2'/>
>>>> <input type='keyboard' bus='ps2'/>
>>>> <hostdev mode='subsystem' type='usb' managed='no'>
>>>> <source>
>>>> <vendor id='0x046d'/>
>>>> <product id='0xc52e'/>
>>>> </source>
>>>> </hostdev>
>>>> <hostdev mode='subsystem' type='usb' managed='no'>
>>>> <source>
>>>> <vendor id='0x28de'/>
>>>> <product id='0x1142'/>
>>>> </source>
>>>> </hostdev>
>>>> <hostdev mode='subsystem' type='usb' managed='no'>
>>>> <source>
>>>> <vendor id='0x0a12'/>
>>>> <product id='0x0001'/>
>>>> </source>
>>>> </hostdev>
>>>> <hostdev mode='subsystem' type='usb' managed='no'>
>>>> <source>
>>>> <vendor id='0x0fcf'/>
>>>> <product id='0x1009'/>
>>>> </source>
>>>> </hostdev>
>>>> <hostdev mode='subsystem' type='usb' managed='no'>
>>>> <source>
>>>> <vendor id='0x1b1c'/>
>>>> <product id='0x1c0b'/>
>>>> </source>
>>>> </hostdev>
>>>> <hostdev mode='subsystem' type='usb' managed='no'>
>>>> <source>
>>>> <vendor id='0x05e3'/>
>>>> <product id='0x0608'/>
>>>> <address bus='1' device='5'/>
>>>> </source>
>>>> </hostdev>
>>>> <hostdev mode='subsystem' type='pci' managed='yes'>
>>>> <source>
>>>> <address domain='0x0000' bus='0x09' slot='0x00'
>>>> function='0x0'
>>> />
>>>> </source>
>>>> <rom bar='on'
>>>> file='/home/virtualguests/windows10/r9290.rom'/>
>>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
>>>> function='0x0' multifunction='on'/>
>>>> </hostdev>
>>>> <hostdev mode='subsystem' type='pci' managed='yes'>
>>>> <source>
>>>> <address domain='0x0000' bus='0x09' slot='0x00'
>>>> function='0x1'/>
>>>> </source>
>>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x09'
>>>> function='0x0'/>
>>>> </hostdev>
>>>> <memballoon model='none'/>
>>>> </devices>
>>>> </domain>
>>>>
>>>>
>>>>
>>>> When I have host-passthrough, Opteron_G5, althon or qemu64 CPUs
>>> configured I
>>>> see a lot of these stack traces just appearing frequently and not
>>>> just
>>> when
>>>> the guest crashes, I see nothing when the guest crashes
>>>>
>>>> [ 2848.156709] ------------[ cut here ]------------
>>>> [ 2848.156719] WARNING: CPU: 0 PID: 1445 at arch/x86/kvm/svm.c:1484
>>>> avic_vcpu_load+0x15a/0x180 [kvm_amd]
>>>> [ 2848.156720] Modules linked in: vhost_net vhost macvtap macvlan
>>>> tun
>>> nfsv3
>>>> rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache hid_logitech_hidpp
>>>> usb_serial_simple cdc_acm usbserial hid_logitech_dj cfg80211 bridge
>>>> stp
>>> llc
>>>> amdgpu sd_mod edac_mce_amd radeon edac_core kvm_amd kvm
>>>> crct10dif_pclmul
>>>> crc32_pclmul crc32c_intel ghash_clmulni_intel pcbc ppdev ttm
>>>> snd_hda_codec_realtek snd_hda_codec_generic drm_kms_helper
>>>> aesni_intel
>>> drm
>>>> btusb snd_hda_intel nls_iso8859_1 aes_x86_64 btrtl crypto_simd
>>>> nls_cp437
>>>> btbcm glue_helper syscopyarea btintel sysfillrect snd_hda_codec
>>>> vfat
>>> r8169
>>>> joydev sysimgblt fat fb_sys_fops i2c_algo_bit bluetooth cryptd
>>>> evdev
>>>> mousedev uas mii input_leds snd_hda_core rfkill pcspkr led_class
>>> snd_hwdep
>>>> mac_hid snd_pcm snd_timer ccp sp5100_tco snd i2c_piix4 soundcore
>>> rng_core
>>>> shpchp wmi parport_pc
>>>> [ 2848.156772] parport fjes 8250_dw i2c_designware_platform
>>> tpm_infineon
>>>> i2c_designware_core button acpi_cpufreq tpm_tis tpm_tis_core tpm
>>>> nfsd
>>>> auth_rpcgss oid_registry nfs_acl lockd grace sch_fq_codel sunrpc
>>> ip_tables
>>>> x_tables ext4 crc16 jbd2 fscrypto mbcache usb_storage hid_generic
>>> usbhid hid
>>>> ahci libahci xhci_pci libata xhci_hcd usbcore scsi_mod nvme
>>>> usb_common
>>>> nvme_core serio vfio_pci irqbypass vfio_virqfd vfio_iommu_type1
>>>> vfio
>>>> [ 2848.156798] CPU: 0 PID: 1445 Comm: CPU 0/KVM Tainted: G W
>>>> 4.10.5-1-ARCH #1
>>>> [ 2848.156798] Hardware name: Gigabyte Technology Co., Ltd. Default
>>>> string/AB350M-Gaming 3-CF, BIOS F2 02/20/2017
>>>> [ 2848.156799] Call Trace:
>>>> [ 2848.156807] dump_stack+0x63/0x83
>>>> [ 2848.156812] __warn+0xcb/0xf0
>>>> [ 2848.156816] warn_slowpath_null+0x1d/0x20
>>>> [ 2848.156819] avic_vcpu_load+0x15a/0x180 [kvm_amd]
>>>> [ 2848.156822] svm_vcpu_unblocking+0x18/0x20 [kvm_amd]
>>>> [ 2848.156834] kvm_vcpu_block+0xd3/0x330 [kvm]
>>>> [ 2848.156844] ? kvm_get_rflags+0x1a/0x30 [kvm]
>>>> [ 2848.156856] kvm_arch_vcpu_ioctl_run+0x4ea/0x1680 [kvm]
>>>> [ 2848.156859] ? _copy_to_user+0x54/0x60
>>>> [ 2848.156867] kvm_vcpu_ioctl+0x339/0x630 [kvm]
>>>> [ 2848.156872] do_vfs_ioctl+0xa3/0x5f0
>>>> [ 2848.156876] ? __fget+0x77/0xb0
>>>> [ 2848.156880] SyS_ioctl+0x79/0x90
>>>> [ 2848.156883] entry_SYSCALL_64_fastpath+0x1a/0xa9
>>>> [ 2848.156885] RIP: 0033:0x7f9980dbd0d7
>>>> [ 2848.156886] RSP: 002b:00007f9972efb8e8 EFLAGS: 00000246
>>>> ORIG_RAX:
>>>> 0000000000000010
>>>> [ 2848.156887] RAX: ffffffffffffffda RBX: 00007f9987e0d001 RCX:
>>>> 00007f9980dbd0d7
>>>> [ 2848.156888] RDX: 0000000000000000 RSI: 000000000000ae80 RDI:
>>>> 0000000000000013
>>>> [ 2848.156888] RBP: 0000000000000001 R08: 000055c65eff4830 R09:
>>>> 00000000000000ff
>>>> [ 2848.156889] R10: 0000000000000001 R11: 0000000000000246 R12:
>>>> 0000000000000001
>>>> [ 2848.156889] R13: 00007f9987e0c000 R14: 0000000000000000 R15:
>>>> 00007f99745a5980
>>>> [ 2848.156905] ---[ end trace e49522bc58864bce ]---
>>>>
>>>> I still haven't tried SeaBIOS yet, I'm still running
>>>> 'pc-i440fx-2.1'
>>> with
>>>> OVMF, not Q35. I couldn't get Windows to boot with Q35.
>>>>
>>>> I also noticed something really odd in the fact that after the
>>>> guest
>>> crashes
>>>> I see random pictures on the TV, which I assume are coming from
>>>> Arch -
>>> I see
>>>> things like a woman at a football stadium and waterfalls - I'm not
>>>> sure
>>> if
>>>> this would be expected if the card is assigned to vfio-pci?
>>>>
>>>> @Steven Walter, can you paste a full copy of your libvirt XML file
>>> please?
>>>>
>>>> Just for completeness here are my IOMMU groups:
>>>>
>>>> [gneville at amdr7 ~]$ lspci -nn
>>>> 00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1450]
>>>> 00:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Device
>>> [1022:1451]
>>>> 00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1452]
>>>> 00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1453]
>>>> 00:01.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1453]
>>>> 00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1452]
>>>> 00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1452]
>>>> 00:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1453]
>>>> 00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1452]
>>>> 00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1452]
>>>> 00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1454]
>>>> 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1452]
>>>> 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1454]
>>>> 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus
>>>> Controller [1022:790b] (rev 59)
>>>> 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH
>>>> LPC
>>> Bridge
>>>> [1022:790e] (rev 51)
>>>> 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1460]
>>>> 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1461]
>>>> 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1462]
>>>> 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1463]
>>>> 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1464]
>>>> 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1465]
>>>> 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1466]
>>>> 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1467]
>>>> 01:00.0 Non-Volatile memory controller [0108]: Samsung Electronics
>>>> Co
>>> Ltd
>>>> Device [144d:a804]
>>>> 03:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:43bb] (rev 02)
>>>> 03:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD]
>>> Device
>>>> [1022:43b7] (rev 02)
>>>> 03:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:43b2] (rev 02)
>>>> 04:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:43b4] (rev 02)
>>>> 04:01.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:43b4] (rev 02)
>>>> 04:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:43b4] (rev 02)
>>>> 05:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd.
>>>> RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
>>>> [10ec:8168]
>>> (rev
>>>> 0c)
>>>> 07:00.0 Non-Volatile memory controller [0108]: Samsung Electronics
>>>> Co
>>> Ltd
>>>> Device [144d:a804]
>>>> 09:00.0 VGA compatible controller [0300]: Advanced Micro Devices,
>>>> Inc.
>>>> [AMD/ATI] Hawaii PRO [Radeon R9 290/390] [1002:67b1]
>>>> 09:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI]
>>> Hawaii
>>>> HDMI Audio [Radeon R9 290/290X / 390/390X] [1002:aac8]
>>>> 11:00.0 Non-Essential Instrumentation [1300]: Advanced Micro
>>>> Devices,
>>> Inc.
>>>> [AMD] Device [1022:145a]
>>>> 11:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc.
>>>> [AMD]
>>>> Device [1022:1456]
>>>> 11:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:145c]
>>>> 12:00.0 Non-Essential Instrumentation [1300]: Advanced Micro
>>>> Devices,
>>> Inc.
>>>> [AMD] Device [1022:1455]
>>>> 12:00.2 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD]
>>>> FCH
>>> SATA
>>>> Controller [AHCI mode] [1022:7901] (rev 51)
>>>> 12:00.3 Audio device [0403]: Advanced Micro Devices, Inc. [AMD]
>>>> Device
>>>> [1022:1457]
>>>> [gneville at amdr7 ~]$
>>>>
>>>> virsh nodedev-dumpxml pci_0000_09_00_0
>>>> <device>
>>>> <name>pci_0000_09_00_0</name>
>>>> <path>/sys/devices/pci0000:00/0000:00:03.1/0000:09:00.0</path>
>>>> <parent>pci_0000_00_03_1</parent>
>>>> <driver>
>>>> <name>vfio-pci</name>
>>>> </driver>
>>>> <capability type='pci'>
>>>> <domain>0</domain>
>>>> <bus>9</bus>
>>>> <slot>0</slot>
>>>> <function>0</function>
>>>> <product id='0x67b1'>Hawaii PRO [Radeon R9 290/390]</product>
>>>> <vendor id='0x1002'>Advanced Micro Devices, Inc.
>>>> [AMD/ATI]</vendor>
>>>> <iommuGroup number='2'>
>>>> <address domain='0x0000' bus='0x00' slot='0x03'
>>>> function='0x0'/>
>>>> <address domain='0x0000' bus='0x09' slot='0x00'
>>>> function='0x1'/>
>>>> <address domain='0x0000' bus='0x00' slot='0x03'
>>>> function='0x1'/>
>>>> <address domain='0x0000' bus='0x09' slot='0x00'
>>>> function='0x0'/>
>>>> </iommuGroup>
>>>> </capability>
>>>> </device>
>>>>
>>>> find /sys/kernel/iommu_groups/ -type l
>>>> /sys/kernel/iommu_groups/7/devices/0000:00:18.6
>>>> /sys/kernel/iommu_groups/7/devices/0000:00:18.4
>>>> /sys/kernel/iommu_groups/7/devices/0000:00:18.2
>>>> /sys/kernel/iommu_groups/7/devices/0000:00:18.0
>>>> /sys/kernel/iommu_groups/7/devices/0000:00:18.7
>>>> /sys/kernel/iommu_groups/7/devices/0000:00:18.5
>>>> /sys/kernel/iommu_groups/7/devices/0000:00:18.3
>>>> /sys/kernel/iommu_groups/7/devices/0000:00:18.1
>>>> /sys/kernel/iommu_groups/5/devices/0000:12:00.2
>>>> /sys/kernel/iommu_groups/5/devices/0000:00:08.1
>>>> /sys/kernel/iommu_groups/5/devices/0000:12:00.0
>>>> /sys/kernel/iommu_groups/5/devices/0000:12:00.3
>>>> /sys/kernel/iommu_groups/5/devices/0000:00:08.0
>>>> /sys/kernel/iommu_groups/3/devices/0000:00:04.0
>>>> /sys/kernel/iommu_groups/1/devices/0000:00:02.0
>>>> /sys/kernel/iommu_groups/6/devices/0000:00:14.0
>>>> /sys/kernel/iommu_groups/6/devices/0000:00:14.3
>>>> /sys/kernel/iommu_groups/4/devices/0000:11:00.2
>>>> /sys/kernel/iommu_groups/4/devices/0000:11:00.0
>>>> /sys/kernel/iommu_groups/4/devices/0000:00:07.1
>>>> /sys/kernel/iommu_groups/4/devices/0000:11:00.3
>>>> /sys/kernel/iommu_groups/4/devices/0000:00:07.0
>>>> /sys/kernel/iommu_groups/2/devices/0000:00:03.0
>>>> /sys/kernel/iommu_groups/2/devices/0000:09:00.1
>>>> /sys/kernel/iommu_groups/2/devices/0000:00:03.1
>>>> /sys/kernel/iommu_groups/2/devices/0000:09:00.0
>>>> /sys/kernel/iommu_groups/0/devices/0000:07:00.0
>>>> /sys/kernel/iommu_groups/0/devices/0000:03:00.1
>>>> /sys/kernel/iommu_groups/0/devices/0000:00:01.3
>>>> /sys/kernel/iommu_groups/0/devices/0000:04:01.0
>>>> /sys/kernel/iommu_groups/0/devices/0000:00:01.1
>>>> /sys/kernel/iommu_groups/0/devices/0000:04:04.0
>>>> /sys/kernel/iommu_groups/0/devices/0000:05:00.0
>>>> /sys/kernel/iommu_groups/0/devices/0000:04:00.0
>>>> /sys/kernel/iommu_groups/0/devices/0000:03:00.2
>>>> /sys/kernel/iommu_groups/0/devices/0000:03:00.0
>>>> /sys/kernel/iommu_groups/0/devices/0000:00:01.0
>>>> /sys/kernel/iommu_groups/0/devices/0000:01:00.0
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Mar 29, 2017 at 12:24 PM, Steven Walter <
>>> stevenrwalter at gmail.com>
>>>> wrote:
>>>>>
>>>>> I got a similar (though multi-GPU) setup working, which I wrote
>>>>> up
>>>>> here:
>>>>> https://www.reddit.com/r/VFIO/comments/616xih/gpu_passthroug
>>> h_with_msi_b350_tomahawk/
>>>>>
>>>>> One thing that may help you is to enable AVIC (kvm_amd.avic=1).
>>>>> What
>>>>> I saw without AVIC was that things would work briefly (only a few
>>>>> seconds for me) before interrupts would stop getting delivered.
>>>>> Sounds like things are working better for you without AVIC than
>>>>> they
>>>>> did for me, but perhaps the extra improvement in IRQ latency would
>>>>> fix
>>>>> the hangs you get during intensive graphics operations.
>>>>>
>>>>>
>>>>> On Tue, Mar 28, 2017 at 6:02 PM, Graham Neville <
>>> grahamneville at gmail.com>
>>>>> wrote:
>>>>>> I've managed to get pci-e passthough working on a gigabyte gaming
>>>>>> 3
>>> matx
>>>>>> MB
>>>>>> and Ryzen 1700, no ACS patch, using only 1 GPU - AMD r9 290.
>>>>>> However
>>> I'm
>>>>>> facing a problem with the whole KVM setup and not sure what it's
>>> related
>>>>>> to.
>>>>>> For the Windows10 guest with the GPU passed through it crashes
>>>>>> (guest
>>>>>> only,
>>>>>> host is fine) whenever I try anything graphics intensive, for
>>>>>> example
>>>>>> running Witcher3. Normal desktop is fine.
>>>>>> Also my Linux guests are acting odd when I try to SSH to them, I
>>> notice
>>>>>> that
>>>>>> the SSH terminals just stop working randomly. And then there's
>>>>>> the
>>> issue
>>>>>> with very slow network throughout to both VMs. I have no idea
>>>>>> what's
>>>>>> going
>>>>>> on. It used to work fine with my Intel setup. There's no logs in
>>> dmesg
>>>>>> to
>>>>>> show a problem either.
>>>>>>
>>>>>> I'm going to try Seabios instead of OVMF to see if I can stop the
>>>>>> crashing.
>>>>>>
>>>>>> Any one having similar issues or anyone can advise?
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> vfio-users mailing list
>>>>>> vfio-users at redhat.com
>>>>>> https://www.redhat.com/mailman/listinfo/vfio-users
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> -Steven Walter <stevenrwalter at gmail.com>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> -Steven Walter <stevenrwalter at gmail.com>
>>>
>>>
>>>
>> _______________________________________________
>> vfio-users mailing list
>> vfio-users at redhat.com
>> https://www.redhat.com/mailman/listinfo/vfio-users
>>
More information about the vfio-users
mailing list