[vfio-users] Windows 10 guest refuses to boot when passed my NVIDIA graphics card

[Jasen Borisov]

I just downgraded to Linux 4.1 on the host ... and the problem went away!
The virtual machine now runs with proper performance. I even booted into a
full XFCE desktop with nvidia proprietary drivers and played some opengl
demos. Then I tried Windows 10, and that worked fine too ... so, in brief,
all the issues were resolved.


This seems like a very serious regression to me. Kernel 4.2 is officially
released and shouldn't be having issues like this. Are the relevant kernel
people who are responsible for the relevant subsystems aware of this?
Should I make an effort to report this?


Thank you for all your help and advice. The issue was really where I least
expected it to be -- the host kernel.


On Wed, Sep 16, 2015 at 1:24 AM, Jasen Borisov <tajjada at gmail.com> wrote:

>
>
> On Wed, Sep 16, 2015 at 1:14 AM, Alex Williamson <
> alex.l.williamson at gmail.com> wrote:
>
>> On Tue, Sep 15, 2015 at 4:09 PM, Jasen Borisov <tajjada at gmail.com> wrote:
>>
>>>
>>>
>>>
>>> On Wed, Sep 16, 2015 at 12:59 AM, Alex Williamson <
>>> alex.l.williamson at gmail.com> wrote:
>>>
>>>> On Tue, Sep 15, 2015 at 3:32 PM, Jasen Borisov <tajjada at gmail.com>
>>>> wrote:
>>>>
>>>>> I managed to successfully boot Arch Linux in the VM. Turned out I
>>>>> wasn't patient enough. With enough waiting, the same VM configuration as
>>>>> before (including my passthrough NVIDIA *and* EHCI!) booted. But it is
>>>>> *incredibly* slow. Took over 10 minutes to get to a login prompt, and a
>>>>> couple of minutes to do its automatic root login. I plugged a USB keyboard
>>>>> into the hardware port on my PC that belongs to the EHCI controller I
>>>>> passed to the VM, and the keyboard worked as expected and I could type
>>>>> inside the VM. I managed to mount a filesystem and dump `dmesg` to a file,
>>>>> so I can get it on my desktop. Here is a pastebin of it:
>>>>> https://bpaste.net/show/e6970c05aff5
>>>>>
>>>>> This makes me think that *maybe* if I leave Windows to boot overnight,
>>>>> next morning I might be surprised by a desktop :)
>>>>>
>>>>> Anyway, it is unacceptably slow, and I would appreciate any help to
>>>>> improve the situation.
>>>>>
>>>>> I noticed a lot of errors regarding msrr, both in the host and guest
>>>>> dmesg, so I guess it would probably be a good idea to add that KVM option
>>>>> you mentioned. However, I doubt that that is the cause of the performance
>>>>> problems. When I remove all VFIO devices and replace them with emulated
>>>>> ones, Arch Linux is very fast (or rather, normal, just very fast in
>>>>> comparison) despite the same msrr errors appearing.
>>>>>
>>>>> Any ideas?
>>>>>
>>>>
>>>> What if you downgrade to kernel 4.1?
>>>>
>>>
>>> As I said, I want to run Windows inside the VM, not Linux. I am just
>>> testing with Linux to see if my VFIO setup is working. I am doing this
>>> because troubleshooting with Linux is a lot easier (due to information from
>>> things like dmesg) than with Windows. Once I know that VFIO is working fine
>>> on my system, I will set up a Windows guest. Hence, kernel
>>> regressions/bugs/features(?) in 4.2 that are not present in 4.1 are not
>>> something I particularly care about. That said, if downgrading to linux 4.1
>>> (not easy, because I was booting Linux from a live iso ... would have to
>>> either install or get an older live iso to do that) would help me figure
>>> out why my VM is slow and would provide worthwhile information, I would be
>>> happy to do it. By now, I have wasted a lot more time waiting for the slow
>>> VM to boot than it would take me to install Linux.
>>>
>>
>> I'm asking about downgrading the *host* to v4.1.
>>
>
> Sorry, my misunderstanding. I could do that. I will try to reboot with
> "kvm.ignore_msrs=1" in my kernel cmdline first, and then try booting 4.1.
>
>
>>
>>
>>>
>>>
>>>> The MTRR issue we've been battling generally resolves itself and the VM
>>>> runs at full speed once passed firmware.  I'll see if I can make my box act
>>>> this poorly.  Running 8 vCPUs on your system is surely not helping
>>>> performance, but this sounds like orders of magnitude worse than that.
>>>>
>>>
>>> Do you mean that having fewer vCPUs will somehow be better for
>>> performance? That seems counterintuitive ... I thought otherwise....
>>>
>>> Well, my physical CPU has 8 physical cores with hyperthreading, so 16
>>> logical cores. I thought it would make sense to give half of that to the VM
>>> and have the rest available for the host. By saying that it is "not helping
>>> performance", do you mean that there is some sort of significant overhead
>>> involved with having more vCPUs?
>>>
>>
>> The XML I see in the first post is using 16 vCPU, leaving none for the
>> host.
>>
>
> I reduced that to 8 vCPU before trying to run Arch Linux, because I
> thought it would be more logical, as I explained above. Sorry for not
> mentioning that change in my VM configuration. I also reduced the memory to
> 4GB, because I thought that allocating so much memory might have had
> something to do with it, but it did not seem to make a difference, so I
> will change it back to 12GB as I had before, once Windows is working.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20150916/4523e342/attachment.htm>