[vfio-users] Fwd: Windows 10 guest refuses to boot when passed my NVIDIA graphics card

Tue Sep 15 20:31:09 UTC 2015

Yes, I have tried passing only the GPU without the EHCI controller. Still,
same problem. My ultimate goal is to run Windows 8.1 or Windows 10 on this
virtual machine with the NVIDIA drivers for gaming. But, since Windows
didn't boot, I decided to try booting Linux for testing, until I am sure
that VFIO works, and then deal with Windows.

I see the Tianocore logo. The EFI shell works fine too.

If I am trying to boot Windows, the Windows logo appears on the screen (but
not the spinning dots loading indicator) and freezes there. Sometimes, the
spinning dots eventually appear, but only for a few seconds, after which
the system resets --> back to Tianocore screen.
If I am trying to boot the Arch Linux liveiso, the Gummiboot menu appears
successfully, but when I select the option to boot Arch Linux, the screen
just goes black and nothing appears after that, so I never see any video
output from the actual Arch Linux system.

I get exactly the same result whenever there is a VFIO / Host PCI device
added to the VM, regardless of whether it is my graphics card, the EHCI
controller, or both. I haven't tried any other devices. Both operating
systems boot perfectly fine when no actual hardware / VFIO devices are
present in the VM configuration.

On Tue, Sep 15, 2015 at 11:09 PM, Alex Williamson <
alex.l.williamson at gmail.com> wrote:

> On Tue, Sep 15, 2015 at 1:57 PM, Jasen Borisov <tajjada at gmail.com> wrote:
>
>>
>> On Tue, Sep 15, 2015 at 10:34 PM, Alex Williamson <
>> alex.l.williamson at gmail.com> wrote:
>>
>>> On Tue, Sep 15, 2015 at 9:32 AM, Jasen Borisov <tajjada at gmail.com>
>>> wrote:
>>>
>>>> Hello everyone,
>>>>
>>>> I have been following the guides on the vfio blogspot for setting up a
>>>> Windows guest for gaming by passing it my NVIDIA GeForce GTX 980 graphics
>>>> card. I have an AMD Radeon R7 250 graphics card for my Linux host. I have
>>>> ran into a strange problem and have not managed to find a solution to it
>>>> online, so I decided to come here and see if anyone knows anything about it.
>>>>
>>>> Here is the problem I am facing:
>>>>
>>>> I installed the Windows guest (Windows 10 Pro) in a kvm virtual machine
>>>> made with virt-manager, using the default qxl/spice setup during the
>>>> installation, as instructed here
>>>> <http://vfio.blogspot.bg/2015/05/vfio-gpu-how-to-series-part-4-our-first.html>.
>>>> After the installation completed, I removed any extra virtual devices and
>>>> added the PCI host/vfio device for my NVIDIA graphics card. I also edited
>>>> the XML to hide the kvm virtualization, in preparation for installing the
>>>> NVIDIA drivers on Windows, when I boot with my actual graphics card. I have
>>>> made no attempt to configure hugepages yet.
>>>>
>>>> However, when starting up the VM, Windows no longer booted
>>>> successfully. I saw the Tianocore splash on my physical monitor, followed
>>>> by the Windows logo. The Windows logo stayed frozen (without the spinning
>>>> loading indicator under it) for a few minutes, and then the spinning-dots
>>>> loading indicator appeared for a few seconds and the machine *reset*. It
>>>> kept resetting again and again, never getting past the Windows boot screen.
>>>>
>>>> I would assume that my PCI passthrough worked successfully, since my
>>>> physical monitor turned on and I saw the video output from the virtual
>>>> machine on it. However, I can't figure out why Windows cannot finish
>>>> booting successfully. Any help with this issue would be greatly appreciated.
>>>>
>>>> The same copy of Windows (or rather, installed from the same
>>>> installation ISO) works fine with my NVIDIA card on my actual hardware /
>>>> when not in a virtual machine, so I am sure this is not a hardware problem.
>>>>
>>>>
>>>> Here is some relevant information about my system:
>>>>
>>>> /proc/cmdline:
>>>> ... intel_iommu=on iommu=pt vfio-pci.ids=10de:13c0,10de:0fbb,8086:8d26
>>>> vfio.disable_vga=1 ...
>>>>
>>>
>>> disable_vga is a vfio-pci option, vfio-pci.disable_vga=1
>>>
>>
>> Ok, thanks, will change that. Not sure if it matters though, since both
>> the host and guest are booting with UEFI.
>>
>
> I think you still want it, but it's probably not going to change anything
> here.
>
>
>> (I omitted my rootfs and other irrelevant kernel options)
>>>> The PCI IDs listed in my kernel commandline are my GPU, its audio, and
>>>> an EHCI controller on my system for USB, in that order.
>>>>
>>>> I did not need to use any "hacks" like the ACS override patch or the
>>>> enable_unsafe_interrupts option, since I did not experience the relevant
>>>> issues that they were made to fix. Each one of the PCI IDs above is in its
>>>> own IOMMU group.
>>>>
>>>
>>> Are you sure about that?  The GPU and audio always share a group as far
>>> as I've seen.
>>>
>>>
>>
>> Ah yes, that is true, the gpu and audio are the same group, but I am
>> passing them both to the VM, so I don't need to split them, hence no need
>> for ACS. What I meant is that my gpu/audio is a separate group from
>> everything else, and so is the EHCI controller I wanted to pass.
>>
>
> Ok, the grouping would be suspect if GPU and audio were actually separate.
>
>
>> My system has no integrated graphics (Core i7 Extreme 5960X CPU).
>>>>
>>>
>>> The XML indicates you're running at least QEMU 2.4, but what kernel
>>> version are you using?  I'll guess at least v4.1 based on vfio commandline
>>> options, but it'd be nice to know.
>>>
>>>
>>
>> Kernel 4.2.0, custom build/config.
>>
>>
>>> These lines appeared in dmesg when the VM started:
>>>>
>>>> [  582.886435] kvm: SMP vm created on host with unstable TSC; guest TSC
>>>> will not be reliable
>>>>
>>>
>>> I'd be a little concerned by this if I were you.
>>>
>>> Why? What problems could it lead to / why is it concerning? Is there
>> anything I can do about it? I googled that message when I first saw it, and
>> I was left with the impression that it is a hardware issue that I can do
>> nothing about, so I just have to deal with it and hope everything will
>> work. I thought the worst thing it can cause is jitter/stuttering in the
>> VM. Perhaps I am wrong?
>>
>>
>>> Are you using "options kvm ignore_msrs=1" in modprobe.d?
>>>
>>
>> No, I am not. Should I be using that?
>>
>> (also, kvm is builtin on my kernel, so, if yes, will have to add that to
>> kernel commandline instead of modprobe.d)
>>
>
> I don't know if it will make a difference here, but I use it.  I think it
> was Passmark PerformanceTest that won't work without it.  I'd be surprised
> to see it affect Linux though.
>
> You say that this boot problem happens even if you only assign the EHCI
> controller, could assigning the EHCI controller be the problem?  Have you
> tried only assigning the GPU?  Where does Linux hang?  Does it get past the
> TianoCore screen?  I've had problems assigning an ASMedia USB3 controller
> where it seemed like it hung in OVMF, but attributed that to the generally
> crappy nature of ASMedia hardware (and an NEC controller worked fine).
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20150915/8f7d2006/attachment.htm>