[vfio-users] problem with passthrough and unsafe interrupts

Thu Sep 17 15:08:56 UTC 2015

UEFI systems have GOP. Every device that provides GOP gets a mirrored
output if not specified by the device driver or the firmware.
VGA devices say that they can decode VGA, and then an arbitration mechanism
comes in - there can be only one device working with VGA at any given time.
If your primary GPU is the one that you wish to pass and the one doing host
VGA - you're screwed, the only way to enable video output for the host is
to load a device driver for the second card. Stuff like Plymouth, frame
buffer console and such break.
And don't even try doing silly things like pressing ctrl-alt-f8 on the host
while the VM is running: you hope to switch to another virtual terminal but
in reality you screw up VM's VGA and GPU's memory.
But if you have a gigabyte motherboard,  you can switch the primary VGA
device, freeing the needed GPU from host's VGA and allowing the host to use
the second card via VGA.

The GT610 is a 1GB silent low profile version from gigabyte. Looks like
it's pretty rare, but it does support UEFI, and if your card doesn't - i
can send you mine.
On Sep 17, 2015 5:54 PM, "Janusz" <januszmk6 at gmail.com> wrote:

> W dniu 17.09.2015 o 11:49, Blank Field pisze:
>
> Gigabyte motherboards allow you to select your primary GPU, but it is only
> relevant to VGA.
> Doing nvidia+amd is good because you can be 100% sure that the host driver
> won't bind the VM GPU.
> I personally have a gt610 paired with hd 7750 and it is totally fine.
>
> why its only relevant to VGA?
> does your gt610 have uefi support? if so, what exactly 610 it is?
>
> On Sep 17, 2015 12:34 PM, "Janusz" <januszmk6 at gmail.com> wrote:
>
>> I think its problem with kernel >=4.2 and ovmf, I cant test it on 4.1
>> because i915 support for skylake there is preliminary and I am getting
>> kernel panic at host boot, so I am thinking about buying cheap another GPU
>> card that will support 3 monitors (2x 1920x1080 + 1x 2560x1440), is this
>> possible to setup that my system will use one card (how can I be sure my
>> motherboard will boot from right GPU?) and I set another one for vga
>> passthrough? will there be a problem if I would have one nvidia and one amd
>> gpu?
>> I include one more time my hw if it is relevant: i7 6700k, msi z170a M7
>> My iommu groups:
>>
>> /sys/kernel/iommu_groups/0/devices/0000:00:00.0
>>
>>
>>> /sys/kernel/iommu_groups/1/devices/0000:00:01.0
>>> /sys/kernel/iommu_groups/1/devices/0000:01:00.0
>>> /sys/kernel/iommu_groups/1/devices/0000:01:00.1
>>>
>> /sys/kernel/iommu_groups/2/devices/0000:00:02.0
>>> /sys/kernel/iommu_groups/3/devices/0000:00:08.0
>>> /sys/kernel/iommu_groups/4/devices/0000:00:14.0
>>> /sys/kernel/iommu_groups/4/devices/0000:00:14.2
>>> /sys/kernel/iommu_groups/5/devices/0000:00:15.0
>>> /sys/kernel/iommu_groups/5/devices/0000:00:15.1
>>> /sys/kernel/iommu_groups/6/devices/0000:00:16.0
>>> /sys/kernel/iommu_groups/7/devices/0000:00:17.0
>>>
>>
>>
>>> /sys/kernel/iommu_groups/8/devices/0000:00:1c.0
>>> /sys/kernel/iommu_groups/8/devices/0000:00:1c.2
>>> /sys/kernel/iommu_groups/8/devices/0000:00:1c.7
>>> /sys/kernel/iommu_groups/8/devices/0000:02:00.0
>>> /sys/kernel/iommu_groups/8/devices/0000:03:00.0
>>> /sys/kernel/iommu_groups/8/devices/0000:04:00.0
>>
>>
>>
>>> /sys/kernel/iommu_groups/9/devices/0000:00:1e.0
>>> /sys/kernel/iommu_groups/10/devices/0000:00:1f.0
>>> /sys/kernel/iommu_groups/10/devices/0000:00:1f.2
>>> /sys/kernel/iommu_groups/10/devices/0000:00:1f.3
>>> /sys/kernel/iommu_groups/10/devices/0000:00:1f.4
>>>
>>> and lspci:
>>>
>>> 00:00.0 Host bridge: Intel Corporation Sky Lake Host Bridge/DRAM
>>> Registers (rev 07)
>>>
>>
>>
>>> 00:01.0 PCI bridge: Intel Corporation Sky Lake PCIe Controller (x16)
>>> (rev 07)
>>>
>>
>>
>>> 00:02.0 VGA compatible controller: Intel Corporation Sky Lake Integrated
>>> Graphics (rev 06)
>>>
>>
>>
>>> 00:08.0 System peripheral: Intel Corporation Sky Lake Gaussian Mixture
>>> Model
>>>
>>
>>
>>> 00:14.0 USB controller: Intel Corporation Sunrise Point-H USB 3.0 xHCI
>>> Controller (rev 31)
>>> 00:14.2 Signal processing controller: Intel Corporation Sunrise Point-H
>>> Thermal subsystem (rev 31)
>>>
>>
>>
>>> 00:15.0 Signal processing controller: Intel Corporation Sunrise Point-H
>>> LPSS I2C Controller #0 (rev 31)
>>> 00:15.1 Signal processing controller: Intel Corporation Sunrise Point-H
>>> LPSS I2C Controller #1 (rev 31)
>>>
>>
>>
>>> 00:16.0 Communication controller: Intel Corporation Sunrise Point-H CSME
>>> HECI #1 (rev 31)
>>>
>>
>>
>>> 00:17.0 SATA controller: Intel Corporation Device a102 (rev 31)
>>>
>>
>>
>>> 00:1c.0 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root
>>> Port #1 (rev f1)
>>> 00:1c.2 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root
>>> Port #3 (rev f1)
>>> 00:1c.7 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root
>>> Port #8 (rev f1)
>>> 00:1e.0 Signal processing controller: Intel Corporation Sunrise Point-H
>>> LPSS UART #0 (rev 31)
>>>
>>
>>
>>> 00:1f.0 ISA bridge: Intel Corporation Sunrise Point-H LPC Controller
>>> (rev 31)
>>> 00:1f.2 Memory controller: Intel Corporation Sunrise Point-H PMC (rev 31)
>>> 00:1f.3 Audio device: Intel Corporation Sunrise Point-H HD Audio (rev 31)
>>> 00:1f.4 SMBus: Intel Corporation Sunrise Point-H SMBus (rev 31)
>>>
>>
>>
>>
>>> 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc.
>>> [AMD/ATI] Hawaii PRO [Radeon R9 290]
>>
>> 01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Device aac8
>>>
>>
>> 02:00.0 USB controller: ASMedia Technology Inc. Device 1242
>> 03:00.0 Ethernet controller: Qualcomm Atheros Device e0a1 (rev 10)
>> 04:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5721
>> Gigabit Ethernet PCI Express (rev 21)
>>
>>
>> W dniu 13.09.2015 o 15:44, Okky Hendriansyah pisze:
>>
>> I assume you're installing with QEMU machine model Q35, have you tried with model i440FX also?
>>
>> Best regards,
>> Okky Hendriansyah
>>
>>
>> On Sep 13, 2015, at 20:15, Janusz <januszmk6 at gmail.com> <januszmk6 at gmail.com> wrote:
>>
>> W dniu 13.09.2015 o 15:08, Okky Hendriansyah pisze:
>>
>> Hi Janus,
>>
>> Are you trying to upgrade to Windows 10? I had issues when upgrading from Windows 8.1 to Windows 10, after changing the cpu to "core2duo", the upgrade finished and I quickly revert back to "host" on my first reboot.
>>
>> Best regards,
>> Okky Hendriansyah
>>
>> no upgrade, I had clean installation on uefi platform done without
>> passing through my gpu, but when I limited threads to 1 windows was
>> unable to boot - every time it reboots it self. I reinstalled windows
>> 10, but still have problem that its sometimes reseting it self while
>> loading (no bsod just reset), and just installed windows 8.1, the same
>> problem. The installation both win10 and win8.1 went without problems.
>>
>>
>> On Sep 13, 2015, at 18:21, Janusz <januszmk6 at gmail.com> <januszmk6 at gmail.com> wrote:
>>
>> W dniu 13.09.2015 o 00:28, Blank Field pisze:
>>
>> Looking at IOMMU groups, his lspci and his problems, there's no way
>> anyone would want to take intel for VFIO usage.
>> Good lord, that's worse than my system...
>>
>> No BSOD, only silent reboot or reset on uefi bios display, also reset
>> issue for gpu (that was already fixed I think in some kernel/qemu
>> version for hawaii, monitor still gets old display sometimes after
>> turining off or reset VM), in dmesg I found only those:
>>
>> [10145.621272] vgaarb: device changed decodes:
>> PCI:0000:01:00.0,olddecodes=io+mem,decodes=io+mem:owns=io+mem
>> [10145.641778] vgaarb: device changed decodes:
>> PCI:0000:01:00.0,olddecodes=io+mem,decodes=io+mem:owns=io+mem
>> [10145.760058] [drm:check_wm_state] *ERROR* mismatch in DDB state
>> pipe A plane 1 (expected (0,0), found (0,289))
>> [10145.760061] [drm:check_wm_state] *ERROR* mismatch in DDB state
>> pipe A cursor (expected (0,0), found (289,297))
>> [10145.760062] [drm:check_wm_state] *ERROR* mismatch in DDB state
>> pipe B plane 1 (expected (0,0), found (297,586))
>> [10145.760063] [drm:check_wm_state] *ERROR* mismatch in DDB state
>> pipe B cursor (expected (0,0), found (586,594))
>> [10148.490876] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19 at 0x270
>> [10148.490881] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x1b at 0x2d0
>> [10154.080574] usb 1-12: reset low-speed USB device number 5 using
>> xhci_hcd
>> [10154.372122] usb 1-12: ep 0x81 - rounding interval to 64
>> microframes, ep desc says 80 microframes
>> [10194.443399] kvm: zapping shadow pages for mmio generation wraparound
>> [10194.453708] kvm: zapping shadow pages for mmio generation wraparound
>> [10165.930150] usb 1-12: ep 0x81 - rounding interval to 64
>> microframes, ep desc says 80 microframes
>> [10168.912066] usb 1-12: reset low-speed USB device number 5 using
>> xhci_hcd
>> [10169.203902] usb 1-12: ep 0x81 - rounding interval to 64
>> microframes, ep desc says 80 microframes
>>
>>
>> I can be missing something as I get lot of warnings from i915 driver
>> (known bug for i915 and skylake), but I did grep for kvm and vfio and
>> didn't find anything else
>>
>> I am running now dev version of qemu because I wanted to test if
>> newer version will give better result (it didn't), and didn't
>> compiled back the stable version yet
>>
>> And yes, I have AMD Catalyst drivers installed in the guest, but as
>> this also happening before it starts to boot windows and when starting
>> windows installation, I don't think this is the reason
>>
>> I think I found some solution. I changed number of threads on cpu from 2
>> to 1, and windows installation is starting every time now. problem still
>> is with installed windows - its rebooting every time it wants to start
>> win10, is windows sensitive for such changes?
>> When I changed number of threads to 2 and number of cores to 3, problem
>> still occurs
>>
>> Also, I upgraded to kernel 4.3-r1
>>
>> I find that someone reported issue to OVMF that it has problems with
>> kernel >= 4.2 https://github.com/tianocore/edk2/issues/21, unfortunately
>> I am not able to test it on 4.1 or 4.0 as on 4.1 I get some weird
>> operations on null pointer in kernel at boot - segfault, and in 4.0 I
>> don't even get signal to monitor - probably hardware is too new for
>> those versions (I know about i915 preliminary support but its not working)
>>
>> anyway, I am gonna test futher later,  and see if with threads=1,cores=4
>> will problem still occur after reinstalling win10
>>
>> _______________________________________________
>> vfio-users mailing listvfio-users at redhat.comhttps://www.redhat.com/mailman/listinfo/vfio-users
>>
>>
>>
>> _______________________________________________
>> vfio-users mailing list
>> vfio-users at redhat.com
>> https://www.redhat.com/mailman/listinfo/vfio-users
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20150917/0b1f7be9/attachment.htm>