[vfio-users] KVM Nvidia Passtrough Code 43 and freezes
Marco Stagge
spiritblackdiamont at gmail.com
Fri Aug 28 18:39:19 UTC 2015
Hmm i need some tuning, the driver just crashed at steam install :o
1 - i know how to do that on xml, but howto on command line?
2 - ah thank didnt notice there are new virtio drivers, thanks :)
greetings
Marco
Am 28.08.2015 um 20:15 schrieb Jon Panozzo:
> Awesome! I think you’ll be happy with your benchmark scores, but to
> improve them further, I’d suggest a few additional tweaks:
>
> 1 - cpuset. you should pin the virtual CPUs in your VM to physical
> threads available on the host (e.g. make vcpu 0 map to pcpu 0). This
> reduces unnecessary overhead caused by context switching and can have
> a pretty big impact to performance in physics benchmarks (this applies
> to gaming).
>
> 2 - I’d also consider using a more current version of the virtio
> drivers. There were a few bugs with Windows 10 discovered that have
> since been patched. The “latest” drivers available from the Fedora
> project are 0.1.109.
>
> Glad you got things working!
>
> - Jon
>
>> On Aug 28, 2015, at 1:11 PM, Marco Stagge
>> <spiritblackdiamont at gmail.com <mailto:spiritblackdiamont at gmail.com>>
>> wrote:
>>
>> Uhm yeah, i dont know how i did this but i think i was the
>> initramfs......
>>
>> i checked lspci -k on my installation, and the 750ti wasnt assigned
>> to pci-stub.....after checking the initramfs-tools/modules, i tried
>> to update the initramfs(i thought i did forget this) and debian throw
>> me a strange error.... after googlin around and solving this problem,
>> i rebootet my system and reinstalled my win10 vm.
>>
>> The devicemanager display no error on the 750ti and i'm on fullhd atm :)
>> The nvidia panel runs without problems, driver is 353.54 :)
>>
>> Well thank you all, guys :)
>> I'll do a benchmark etc now, yay ^_^
>>
>> greetings
>> Marco
>>
>> Am 28.08.2015 um 19:31 schrieb Jon Panozzo:
>>> Hmm. Can you ssh into the system and type lspci -k and copy/paste
>>> the results back here? This is very odd behavior indeed.
>>>
>>> One other thing we can try too is to pci-stub this device and
>>> reboot. Not sure how you were doing it on your system, but with
>>> unRAID, you do this by logging in via ssh, then running the command
>>> lspci -n. Locate the row for your device (the 02:00.0 and .1), then
>>> copy the vendor/product ID.
>>>
>>> Back on the webGui, go to the Main tab, then click on the word
>>> “Flash” under the devices column. Now go to the syslinux section
>>> and modify the /append line. Here’s mine, where I stub numerous
>>> devices:
>>>
>>> default /syslinux/menu.c32
>>> menu title Lime Technology
>>> prompt 0
>>> timeout 50
>>> label unRAID OS
>>> menu default
>>> kernel /bzimage
>>> append
>>> pci-stub.ids=1002:68f9,1002:aa68,10de:1004,10de:0e1a,1a0a:6202,8086:0412
>>> pcie_acs_override=downstream initrd=/bzroot
>>> label unRAID OS Safe Mode (no plugins)
>>> kernel /bzimage
>>> append initrd=/bzroot unraidsafemode
>>> label Memtest86+
>>> kernel /memtest
>>> label Xen/unRAID OS
>>> kernel /syslinux/mboot.c32
>>> append /xen dom0_mem=4096M iommu=on --- /bzimage
>>> xen-pciback.hide=(01:00.0)(01:00.1)(02:00.0)(02:00.1) intel_iommu=on
>>> --- /bzroot
>>> label Xen/unRAID OS Safe Mode (no plugins)
>>> kernel /syslinux/mboot.c32
>>> append /xen --- /bzimage --- /bzroot unraidsafemode
>>>
>>> You shouldn’t need to stub the devices in unRAID because we don’t
>>> load GPU drivers, so nothing should be bound to the device.
>>>
>>> If neither of these work, you may have me stumped :-(
>>>
>>> As a last resort, you can go to Tools -> Diagnostics and click
>>> collect, then send me that zip file for further analysis.
>>>
>>> - Jon
>>>
>>>> On Aug 28, 2015, at 12:13 PM, Marco Stagge
>>>> <spiritblackdiamont at gmail.com
>>>> <mailto:spiritblackdiamont at gmail.com>> wrote:
>>>>
>>>> okay here:
>>>>
>>>> /sys/kernel/iommu_groups/0/devices/0000:00:00.0
>>>> /sys/kernel/iommu_groups/1/devices/0000:00:02.0
>>>> /sys/kernel/iommu_groups/2/devices/0000:00:03.0
>>>> /sys/kernel/iommu_groups/3/devices/0000:00:04.0
>>>> /sys/kernel/iommu_groups/4/devices/0000:00:05.0
>>>> /sys/kernel/iommu_groups/5/devices/0000:00:06.0
>>>> /sys/kernel/iommu_groups/6/devices/0000:00:07.0
>>>> /sys/kernel/iommu_groups/7/devices/0000:00:09.0
>>>> /sys/kernel/iommu_groups/8/devices/0000:00:0a.0
>>>> /sys/kernel/iommu_groups/9/devices/0000:00:11.0
>>>> /sys/kernel/iommu_groups/10/devices/0000:00:12.0
>>>> /sys/kernel/iommu_groups/10/devices/0000:00:12.2
>>>> /sys/kernel/iommu_groups/11/devices/0000:00:13.0
>>>> /sys/kernel/iommu_groups/11/devices/0000:00:13.2
>>>> /sys/kernel/iommu_groups/12/devices/0000:00:14.0
>>>> /sys/kernel/iommu_groups/13/devices/0000:00:14.2
>>>> /sys/kernel/iommu_groups/14/devices/0000:00:14.3
>>>> /sys/kernel/iommu_groups/15/devices/0000:00:14.4
>>>> /sys/kernel/iommu_groups/15/devices/0000:09:06.0
>>>> /sys/kernel/iommu_groups/16/devices/0000:00:14.5
>>>> /sys/kernel/iommu_groups/17/devices/0000:00:15.0
>>>> /sys/kernel/iommu_groups/18/devices/0000:00:16.0
>>>> /sys/kernel/iommu_groups/18/devices/0000:00:16.2
>>>> /sys/kernel/iommu_groups/19/devices/0000:01:00.0
>>>> /sys/kernel/iommu_groups/19/devices/0000:01:00.1
>>>> /sys/kernel/iommu_groups/20/devices/0000:02:00.0
>>>> /sys/kernel/iommu_groups/20/devices/0000:02:00.1
>>>> /sys/kernel/iommu_groups/21/devices/0000:03:00.0
>>>> /sys/kernel/iommu_groups/22/devices/0000:04:00.0
>>>> /sys/kernel/iommu_groups/23/devices/0000:05:00.0
>>>> /sys/kernel/iommu_groups/24/devices/0000:06:00.0
>>>> /sys/kernel/iommu_groups/25/devices/0000:07:00.0
>>>> /sys/kernel/iommu_groups/26/devices/0000:08:00.0
>>>> /sys/kernel/iommu_groups/27/devices/0000:0a:00.0
>>>> /sys/kernel/iommu_groups/27/devices/0000:0a:00.1
>>>>
>>>>
>>>> Am 28.08.2015 um 19:11 schrieb Jon Panozzo:
>>>>> Under Tools -> System Devices, please copy and paste the IOMMU
>>>>> groups section back in an e-mail to me.
>>>>>
>>>>> - Jon
>>>>>
>>>>>> On Aug 28, 2015, at 12:09 PM, Marco Stagge
>>>>>> <spiritblackdiamont at gmail.com
>>>>>> <mailto:spiritblackdiamont at gmail.com>> wrote:
>>>>>>
>>>>>> done, but still the same error :(
>>>>>>
>>>>>> Am 28.08.2015 um 18:59 schrieb Jon Panozzo:
>>>>>>> Ok, that is because you need to enable PCI ACS Override. Under
>>>>>>> Settings -> VM Manager, enable PCI ACS Override. Then reboot
>>>>>>> your system.
>>>>>>>
>>>>>>>> On Aug 28, 2015, at 11:57 AM, Marco Stagge
>>>>>>>> <spiritblackdiamont at gmail.com> wrote:
>>>>>>>>
>>>>>>>> Hi Jon,
>>>>>>>>
>>>>>>>> i've tried to create a vm but its not possible:
>>>>>>>> Error creating VM: internal error: early end of file from
>>>>>>>> monitor: possible problem:
>>>>>>>> 2015-08-28T16:54:50.210851Z qemu-system-x86_64: -device
>>>>>>>> vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on:
>>>>>>>> vfio: failed to set iommu for container: Operation not permitted
>>>>>>>> 2015-08-28T16:54:50.210877Z qemu-system-x86_64: -device
>>>>>>>> vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on:
>>>>>>>> vfio: failed to setup container for group 20
>>>>>>>> 2015-08-28T16:54:50.210886Z qemu-system-x86_64: -device
>>>>>>>> vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on:
>>>>>>>> vfio: failed to get group 20
>>>>>>>> 2015-08-28T16:54:50.210898Z qemu-system-x86_64: -device
>>>>>>>> vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on:
>>>>>>>> Device initialization failed
>>>>>>>> 2015-08-28T16:54:50.210910Z qemu-system-x86_64: -device
>>>>>>>> vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on:
>>>>>>>> Device 'vfio-pci' could not be initialized
>>>>>>>>
>>>>>>>> did i missed something?
>>>>>>>>
>>>>>>>> greetings
>>>>>>>> Marco
>>>>>>>>
>>>>>>>> Am 28.08.2015 um 18:05 schrieb Jon Panozzo:
>>>>>>>>> Hi Marco,
>>>>>>>>>
>>>>>>>>> Not sure if you want to go down this road or not, but a
>>>>>>>>> relatively quick path to checking to see if this is a
>>>>>>>>> kernel/OS config issue would be to load up our OS (unRAID)
>>>>>>>>> which boots of a USB stick and see if you have success there.
>>>>>>>>> We’ve tuned our OS for use with VFIO/VGA assignment, and we
>>>>>>>>> don’t load a desktop GUI at all (nor any graphics devices).
>>>>>>>>> The OS uses libvirt and provides a webGui for testing various
>>>>>>>>> configs (OVMF vs. SeaBIOS and i440fx vs. Q35). Hopefully this
>>>>>>>>> isn’t seen as a shameless plug on our OS here, but it could be
>>>>>>>>> a way to narrow down the issue, and it’s free to try
>>>>>>>>> (http://lime-technology.com).
>>>>>>>>>
>>>>>>>>> I can tell you that the NVIDIA drivers have nothing to do with
>>>>>>>>> it. I have multiple NVIDIA GPUs that are running on multiple
>>>>>>>>> systems. So long as you don’t enable the hyper-v
>>>>>>>>> enlightenments through QEMU/KVM and specify kvm=off like you
>>>>>>>>> are, the card does not detect that it is running in a virtual
>>>>>>>>> machine / render the code 43 error. There are other issues
>>>>>>>>> that can cause code 43 unrelated to detection by the device
>>>>>>>>> drivers, but narrowing down the specifics can be a challenge.
>>>>>>>>>
>>>>>>>>> - Jon
>>>>>>>>>
>>>>>>>>>> On Aug 28, 2015, at 10:58 AM, Marco Stagge
>>>>>>>>>> <spiritblackdiamont at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Jonathan,
>>>>>>>>>>
>>>>>>>>>> this is a AMD Board without a integrated GPU :)
>>>>>>>>>>
>>>>>>>>>> Yep, nouveau driver don't load on this card but the nouveau
>>>>>>>>>> driver tells me on boot some strange lines at dmesg but can
>>>>>>>>>> display the EFI Screen etc.
>>>>>>>>>>
>>>>>>>>>> Yeah i saw on the bbs from archlinux some guys with nvidia
>>>>>>>>>> cards running, but this thread is a bit "old". So i think
>>>>>>>>>> nvidia changed the driver to recognize the machine as
>>>>>>>>>> virtual(Windows 10 does).
>>>>>>>>>>
>>>>>>>>>> Without the romfile, it wont display anything(same on the
>>>>>>>>>> radeon cards) :o
>>>>>>>>>>
>>>>>>>>>> Here's my lspci -k:
>>>>>>>>>> 00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> RD890 PCI to PCI bridge (external gfx0 port B) (rev 02)
>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> RD890 PCI to PCI bridge (external gfx0 port B)
>>>>>>>>>> 00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD/ATI] RD990
>>>>>>>>>> I/O Memory Management Unit (IOMMU)
>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> RD990 I/O Memory Management Unit (IOMMU)
>>>>>>>>>> 00:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> RD890 PCI to PCI bridge (PCI express gpp port B)
>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>> 00:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> RD890 PCI to PCI bridge (PCI express gpp port C)
>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>> 00:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> RD890 PCI to PCI bridge (PCI express gpp port D)
>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>> 00:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> RD890 PCI to PCI bridge (PCI express gpp port E)
>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>> 00:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> RD890 PCI to PCI bridge (PCI express gpp port F)
>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>> 00:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> RD890 PCI to PCI bridge (PCI express gpp port G)
>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>> 00:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> RD890 PCI to PCI bridge (PCI express gpp port H)
>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>> 00:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> RD890 PCI to PCI bridge (external gfx1 port A)
>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>> 00:11.0 SATA controller: Advanced Micro Devices, Inc.
>>>>>>>>>> [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] (rev 40)
>>>>>>>>>> Subsystem: ASUSTeK Computer Inc. Device 84dd
>>>>>>>>>> Kernel driver in use: ahci
>>>>>>>>>> 00:12.0 USB controller: Advanced Micro Devices, Inc.
>>>>>>>>>> [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>>> Kernel driver in use: ohci-pci
>>>>>>>>>> 00:12.2 USB controller: Advanced Micro Devices, Inc.
>>>>>>>>>> [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>>> Kernel driver in use: ehci-pci
>>>>>>>>>> 00:13.0 USB controller: Advanced Micro Devices, Inc.
>>>>>>>>>> [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>> 00:13.2 USB controller: Advanced Micro Devices, Inc.
>>>>>>>>>> [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>> 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00
>>>>>>>>>> SMBus Controller (rev 42)
>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> SBx00 SMBus Controller
>>>>>>>>>> Kernel driver in use: piix4_smbus
>>>>>>>>>> 00:14.2 Audio device: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> SBx00 Azalia (Intel HDA) (rev 40)
>>>>>>>>>> Subsystem: ASUSTeK Computer Inc. M4A89GTD PRO/USB3
>>>>>>>>>> Motherboard
>>>>>>>>>> 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> SB7x0/SB8x0/SB9x0 LPC host controller (rev 40)
>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> SB7x0/SB8x0/SB9x0 LPC host controller
>>>>>>>>>> 00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> SBx00 PCI to PCI Bridge (rev 40)
>>>>>>>>>> 00:14.5 USB controller: Advanced Micro Devices, Inc.
>>>>>>>>>> [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
>>>>>>>>>> Kernel driver in use: ohci-pci
>>>>>>>>>> 00:15.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0)
>>>>>>>>>> Kernel driver in use: pcieport
>>>>>>>>>> 00:16.0 USB controller: Advanced Micro Devices, Inc.
>>>>>>>>>> [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>> 00:16.2 USB controller: Advanced Micro Devices, Inc.
>>>>>>>>>> [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>>> Subsystem: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> SB7x0/SB8x0/SB9x0 USB EHCI Controller
>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>> 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>>>>>>>> Family 15h Processor Function 0
>>>>>>>>>> 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>>>>>>>> Family 15h Processor Function 1
>>>>>>>>>> 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>>>>>>>> Family 15h Processor Function 2
>>>>>>>>>> 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>>>>>>>> Family 15h Processor Function 3
>>>>>>>>>> Kernel driver in use: k10temp
>>>>>>>>>> 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>>>>>>>> Family 15h Processor Function 4
>>>>>>>>>> Kernel driver in use: fam15h_power
>>>>>>>>>> 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD]
>>>>>>>>>> Family 15h Processor Function 5
>>>>>>>>>> 01:00.0 VGA compatible controller: NVIDIA Corporation GF119
>>>>>>>>>> [GeForce GT 520] (rev a1)
>>>>>>>>>> Subsystem: ZOTAC International (MCO) Ltd. Device 3214
>>>>>>>>>> Kernel driver in use: nouveau
>>>>>>>>>> 01:00.1 Audio device: NVIDIA Corporation GF119 HDMI Audio
>>>>>>>>>> Controller (rev a1)
>>>>>>>>>> Subsystem: ZOTAC International (MCO) Ltd. Device 3214
>>>>>>>>>> 02:00.0 VGA compatible controller: NVIDIA Corporation GM107
>>>>>>>>>> [GeForce GTX 750 Ti] (rev a2)
>>>>>>>>>> Subsystem: eVga.com <http://evga.com/>. Corp. Device 3753
>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>> 02:00.1 Audio device: NVIDIA Corporation Device 0fbc (rev a1)
>>>>>>>>>> Subsystem: eVga.com <http://evga.com/>. Corp. Device 3753
>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>> 03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
>>>>>>>>>> RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
>>>>>>>>>> (rev 06)
>>>>>>>>>> Subsystem: ASUSTeK Computer Inc. P8P67 and other
>>>>>>>>>> motherboards
>>>>>>>>>> Kernel driver in use: r8169
>>>>>>>>>> 04:00.0 USB controller: ASMedia Technology Inc. ASM1042
>>>>>>>>>> SuperSpeed USB Host Controller
>>>>>>>>>> Subsystem: ASUSTeK Computer Inc. P8B WS Motherboard
>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>> 05:00.0 SATA controller: JMicron Technology Corp. JMB362 SATA
>>>>>>>>>> Controller (rev 10)
>>>>>>>>>> Subsystem: ASUSTeK Computer Inc. P8P67 Deluxe Motherboard
>>>>>>>>>> Kernel driver in use: ahci
>>>>>>>>>> 06:00.0 USB controller: ASMedia Technology Inc. ASM1042
>>>>>>>>>> SuperSpeed USB Host Controller
>>>>>>>>>> Subsystem: ASUSTeK Computer Inc. P8B WS Motherboard
>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>> 07:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd.
>>>>>>>>>> RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
>>>>>>>>>> (rev 06)
>>>>>>>>>> Subsystem: Device 7470:3468
>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>> 08:00.0 SATA controller: JMicron Technology Corp. JMB362 SATA
>>>>>>>>>> Controller (rev 10)
>>>>>>>>>> Subsystem: ASUSTeK Computer Inc. P8P67 Deluxe Motherboard
>>>>>>>>>> Kernel driver in use: ahci
>>>>>>>>>> 09:06.0 FireWire (IEEE 1394): VIA Technologies, Inc.
>>>>>>>>>> VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller (rev c0)
>>>>>>>>>> Subsystem: ASUSTeK Computer Inc. Motherboard
>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>> 0a:00.0 VGA compatible controller: Advanced Micro Devices,
>>>>>>>>>> Inc. [AMD/ATI] Turks XT [Radeon HD 6670/7670]
>>>>>>>>>> Subsystem: PC Partner Limited / Sapphire Technology
>>>>>>>>>> Device e195
>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>> 0a:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI]
>>>>>>>>>> Turks/Whistler HDMI Audio [Radeon HD 6000 Series]
>>>>>>>>>> Subsystem: PC Partner Limited / Sapphire Technology
>>>>>>>>>> Device aa90
>>>>>>>>>> Kernel driver in use: vfio-pci
>>>>>>>>>>
>>>>>>>>>> greetings
>>>>>>>>>> Marco - Spirit
>>>>>>>>>>
>>>>>>>>>> Am 28.08.2015 um 17:49 schrieb Jon Panozzo:
>>>>>>>>>>> Hi Marco,
>>>>>>>>>>>
>>>>>>>>>>> A few suggestions:
>>>>>>>>>>>
>>>>>>>>>>> In your motherboard BIOS, do you have an option for primary
>>>>>>>>>>> graphics to be toggled between on-board and PCIe? If so,
>>>>>>>>>>> please make sure that it’s set to NOT utilize the graphics
>>>>>>>>>>> device you’re trying to use with the VM. I’ve seen issues
>>>>>>>>>>> with that consistently (most result in no GPU pass through
>>>>>>>>>>> at all, but with some hardware, it may result in what you’re
>>>>>>>>>>> experiencing).
>>>>>>>>>>>
>>>>>>>>>>> Also, are you using pci-stub to prevent the GPU from having
>>>>>>>>>>> a nouveau driver loaded before starting your VM? If not,
>>>>>>>>>>> this may also impact your experience.
>>>>>>>>>>>
>>>>>>>>>>> I know we have several users passing through a GTX 750
>>>>>>>>>>> without difficulty, and without needing to load the rom file
>>>>>>>>>>> as you are below.
>>>>>>>>>>>
>>>>>>>>>>> Hope this helps!
>>>>>>>>>>>
>>>>>>>>>>> Best Regards,
>>>>>>>>>>>
>>>>>>>>>>> Jonathan Panozzo
>>>>>>>>>>>
>>>>>>>>>>>> On Aug 28, 2015, at 10:42 AM, Marco Stagge
>>>>>>>>>>>> <spiritblackdiamont at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hello @all,
>>>>>>>>>>>>
>>>>>>>>>>>> at first, my english is not that good :/ so sorry for wrong
>>>>>>>>>>>> spelling etc :o
>>>>>>>>>>>>
>>>>>>>>>>>> I've tried bunches of configs and hardware with kvm and
>>>>>>>>>>>> passtrough, with Radeon Cards there are no problems but
>>>>>>>>>>>> with Nvidia ...
>>>>>>>>>>>> it look like its nearly impossible ... it works to Display
>>>>>>>>>>>> TianoCore bootscreen and Windows Install etc, but after
>>>>>>>>>>>> installing the driver the whole machine freezes and on the
>>>>>>>>>>>> console i got the message "kernel: disabling irq #20" wich
>>>>>>>>>>>> is my GTX 750 Ti.
>>>>>>>>>>>>
>>>>>>>>>>>> the strange thingy is "x-vga=off" is the only way to see
>>>>>>>>>>>> something on the monitor(i thought it should be "on" for
>>>>>>>>>>>> this)....
>>>>>>>>>>>>
>>>>>>>>>>>> I've tried this with q35 chipset and the fx440, with
>>>>>>>>>>>> seabios and TianoCore EFI, Windows 7 and Windows 10(dont
>>>>>>>>>>>> have Win8 or want it ..)
>>>>>>>>>>>>
>>>>>>>>>>>> Here my config and specs
>>>>>>>>>>>>
>>>>>>>>>>>> /root/qemu-2.4.0/x86_64-softmmu/qemu-system-x86_64
>>>>>>>>>>>> -enable-kvm -M q35 -m 8G -cpu host,kvm=off \
>>>>>>>>>>>> -smp 8,sockets=1,cores=8,threads=1 \
>>>>>>>>>>>> -vga none -k de -rtc base=localtime -realtime mlock=on \
>>>>>>>>>>>> -drive if=pflash,format=raw,file=/root/OVMF-pure-efi.fd \
>>>>>>>>>>>> -monitor stdio \
>>>>>>>>>>>> -debugcon file:/dev/stderr -global
>>>>>>>>>>>> isa-debugcon.iobase=0x402 \
>>>>>>>>>>>> -device virtio-scsi-pci,id=scsi \
>>>>>>>>>>>> -device
>>>>>>>>>>>> vfio-pci,host=02:00.0,rombar=1,x-vga=off,romfile=/root/GTX-750Ti.rom
>>>>>>>>>>>> \
>>>>>>>>>>>> -device vfio-pci,host=02:00.1 \
>>>>>>>>>>>> -device vfio-pci,host=04:00.0 \
>>>>>>>>>>>> -device vfio-pci,host=00:13.0 \
>>>>>>>>>>>> -device vfio-pci,host=00:13.2 \
>>>>>>>>>>>> -drive file=/dev/sdb,if=virtio,cache=none,discard=on \
>>>>>>>>>>>> -drive
>>>>>>>>>>>> file=/media/syno/Programme/windows/Windows_7_SP1_DE_AIO_X64.iso,id=isocd,if=none
>>>>>>>>>>>> \
>>>>>>>>>>>> -device scsi-cd,drive=isocd \
>>>>>>>>>>>> -cdrom /media/syno/Programme/Linux/virtio-win-0.1-100.iso \
>>>>>>>>>>>> -usbdevice host:04b4:8328 \
>>>>>>>>>>>> -global isa-fdc.driveA=
>>>>>>>>>>>>
>>>>>>>>>>>> AMD FX8350
>>>>>>>>>>>> ASUS M5A99X EVO
>>>>>>>>>>>> 32GB Kingston Value RAM
>>>>>>>>>>>> AMD Radeon HD6670(guest) + Nvidia GTX 750 Ti(guest) +
>>>>>>>>>>>> GT520(Host)
>>>>>>>>>>>> SSD now 120GB(guest) + Mushkin Chronos Deluxe 60GB(guest)
>>>>>>>>>>>>
>>>>>>>>>>>> interrupts:
>>>>>>>>>>>> CPU0 CPU1 CPU2 CPU3 CPU4
>>>>>>>>>>>> CPU5 CPU6 CPU7
>>>>>>>>>>>> 0: 47 0 0 0 0
>>>>>>>>>>>> 0 0 0 IR-IO-APIC-edge timer
>>>>>>>>>>>> 1: 1 0 0 0 0
>>>>>>>>>>>> 0 0 1 IR-IO-APIC-edge i8042
>>>>>>>>>>>> 8: 0 0 0 0 0
>>>>>>>>>>>> 0 1 0 IR-IO-APIC-edge rtc0
>>>>>>>>>>>> 9: 0 0 0 0 0
>>>>>>>>>>>> 0 0 0 IR-IO-APIC-fasteoi acpi
>>>>>>>>>>>> 12: 0 1 0 1 0
>>>>>>>>>>>> 2 0 0 IR-IO-APIC-edge i8042
>>>>>>>>>>>> 17: 97731 573202 44343 68863 449791
>>>>>>>>>>>> 278946 766521 145166 IR-IO-APIC 17-fasteoi
>>>>>>>>>>>> ehci_hcd:usb1, vfio-intx(0000:0a:00.1)
>>>>>>>>>>>> 18: 6667 21540 5559 6748 24441
>>>>>>>>>>>> 16748 4959 8611 IR-IO-APIC 18-fasteoi
>>>>>>>>>>>> ohci_hcd:usb8, ohci_hcd:usb10
>>>>>>>>>>>> 19: 22052 93315 44652 67664 119162
>>>>>>>>>>>> 75922 85567 23549 IR-IO-APIC 19-fasteoi
>>>>>>>>>>>> 0000:00:11.0
>>>>>>>>>>>> 22: 0 3 1 8 2
>>>>>>>>>>>> 1 1 1 IR-IO-APIC 22-fasteoi
>>>>>>>>>>>> vfio-intx(0000:00:16.0)
>>>>>>>>>>>> 23: 10599 70971 15651 26344 78321
>>>>>>>>>>>> 47939 232415 36216 IR-IO-APIC 23-fasteoi
>>>>>>>>>>>> vfio-intx(0000:00:16.2)
>>>>>>>>>>>> 27: 0 0 0 0 0
>>>>>>>>>>>> 0 0 0 IR-PCI-MSI-edge AMD-Vi
>>>>>>>>>>>> 30: 859 2247 3874003 745023 1000
>>>>>>>>>>>> 739 1462 145680 IR-PCI-MSI-edge eth0
>>>>>>>>>>>> 33: 627986 4106418 3829 6174 1615
>>>>>>>>>>>> 953 140 156 IR-PCI-MSI-edge
>>>>>>>>>>>> vfio-msi[0](0000:0a:00.0)
>>>>>>>>>>>> 34: 43 740 54 294 43
>>>>>>>>>>>> 4483073 43 1708 IR-PCI-MSI-edge
>>>>>>>>>>>> vfio-msix[0](0000:07:00.0)
>>>>>>>>>>>> 35: 2 1 1 70 6
>>>>>>>>>>>> 66 1 22716 IR-PCI-MSI-edge
>>>>>>>>>>>> vfio-msix[1](0000:07:00.0)
>>>>>>>>>>>> 36: 41 1 0 6 3685144
>>>>>>>>>>>> 2 118 2 IR-PCI-MSI-edge
>>>>>>>>>>>> vfio-msix[2](0000:07:00.0)
>>>>>>>>>>>> 37: 1 1 20027 0 39
>>>>>>>>>>>> 0 257 1 IR-PCI-MSI-edge
>>>>>>>>>>>> vfio-msix[3](0000:07:00.0)
>>>>>>>>>>>> 38: 2617 16082 450 755 123855
>>>>>>>>>>>> 75581 72374 11818 IR-PCI-MSI-edge
>>>>>>>>>>>> vfio-msix[0](0000:06:00.0)
>>>>>>>>>>>> 39: 0 0 0 0 0
>>>>>>>>>>>> 0 0 0 IR-PCI-MSI-edge
>>>>>>>>>>>> vfio-msix[1](0000:06:00.0)
>>>>>>>>>>>> 40: 0 0 0 0 0
>>>>>>>>>>>> 0 0 0 IR-PCI-MSI-edge
>>>>>>>>>>>> vfio-msix[2](0000:06:00.0)
>>>>>>>>>>>> 41: 0 0 0 0 0
>>>>>>>>>>>> 0 0 0 IR-PCI-MSI-edge
>>>>>>>>>>>> vfio-msix[3](0000:06:00.0)
>>>>>>>>>>>> 42: 0 0 0 0 0
>>>>>>>>>>>> 0 0 0 IR-PCI-MSI-edge
>>>>>>>>>>>> vfio-msix[4](0000:06:00.0)
>>>>>>>>>>>> 43: 0 0 0 0 0
>>>>>>>>>>>> 0 0 0 IR-PCI-MSI-edge
>>>>>>>>>>>> vfio-msix[5](0000:06:00.0)
>>>>>>>>>>>> 44: 0 0 0 0 0
>>>>>>>>>>>> 0 0 0 IR-PCI-MSI-edge
>>>>>>>>>>>> vfio-msix[6](0000:06:00.0)
>>>>>>>>>>>> 45: 0 0 0 0 0
>>>>>>>>>>>> 0 0 0 IR-PCI-MSI-edge
>>>>>>>>>>>> vfio-msix[7](0000:06:00.0)
>>>>>>>>>>>> 51: 4115 5167 5148 5828 7122
>>>>>>>>>>>> 5740 20944 7389 IR-IO-APIC 27-fasteoi
>>>>>>>>>>>> 0000:05:00.0
>>>>>>>>>>>> 52: 0 0 0 0 0
>>>>>>>>>>>> 0 0 0 IR-IO-APIC 23-fasteoi
>>>>>>>>>>>> 0000:08:00.0
>>>>>>>>>>>> 54: 70872 71659 71958 71508 71437
>>>>>>>>>>>> 72232 75968 115521 IR-PCI-MSI-edge nouveau
>>>>>>>>>>>> 56: 5738 5576 5715 5605 5551
>>>>>>>>>>>> 5403 5792 10014 IR-PCI-MSI-edge nouveau
>>>>>>>>>>>> NMI: 535 629 505 409 474
>>>>>>>>>>>> 508 457 388 Non-maskable interrupts
>>>>>>>>>>>> LOC: 7873279 2105376 6840648 3347764 5748321
>>>>>>>>>>>> 2139094 7856471 3305521 Local timer interrupts
>>>>>>>>>>>> SPU: 0 0 0 0 0
>>>>>>>>>>>> 0 0 0 Spurious interrupts
>>>>>>>>>>>> PMI: 535 629 505 409 474
>>>>>>>>>>>> 508 457 388 Performance monitoring
>>>>>>>>>>>> interrupts
>>>>>>>>>>>> IWI: 0 0 0 0 0
>>>>>>>>>>>> 1 0 0 IRQ work interrupts
>>>>>>>>>>>> RTR: 0 0 0 0 0
>>>>>>>>>>>> 0 0 0 APIC ICR read retries
>>>>>>>>>>>> RES: 31834176 46456464 40752536 13038581 29382846
>>>>>>>>>>>> 31897150 34234584 16740944 Rescheduling interrupts
>>>>>>>>>>>> CAL: 2026 1990 2018 2081 2046
>>>>>>>>>>>> 1441 2141 2063 Function call interrupts
>>>>>>>>>>>> TLB: 3857 3712 3424 6321 4082
>>>>>>>>>>>> 4559 3763 6583 TLB shootdowns
>>>>>>>>>>>> TRM: 0 0 0 0 0
>>>>>>>>>>>> 0 0 0 Thermal event interrupts
>>>>>>>>>>>> THR: 0 0 0 0 0
>>>>>>>>>>>> 0 0 0 Threshold APIC interrupts
>>>>>>>>>>>> MCE: 0 0 0 0 0
>>>>>>>>>>>> 0 0 0 Machine check exceptions
>>>>>>>>>>>> MCP: 83 83 83 83 83
>>>>>>>>>>>> 83 83 83 Machine check polls
>>>>>>>>>>>> HYP: 0 0 0 0 0
>>>>>>>>>>>> 0 0 0 Hypervisor callback
>>>>>>>>>>>> interrupts
>>>>>>>>>>>> ERR: 0
>>>>>>>>>>>> MIS: 0
>>>>>>>>>>>>
>>>>>>>>>>>> any ideas?
>>>>>>>>>>>>
>>>>>>>>>>>> greetings
>>>>>>>>>>>> Spirit
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> vfio-users mailing list
>>>>>>>>>>>> vfio-users at redhat.com
>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/vfio-users
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20150828/4e4d24b0/attachment.htm>
More information about the vfio-users
mailing list