[vfio-users] Guest does not boot OS with GPU passed through, stuck at TianoCore boot splash

[Stefan Seil]

Thank you for your detailed response!

So if I understand you correctly, you are saying that from now on the 
issue lies within the guest OS, and thus I am rather limited in what I 
can do on my end. The good news is that I have actually gotten it to 
work with a Windows 10 guest. Running Windows was my actual goal, so not 
being able to pass the GPU to a Linux guest is not the end of the world 
for me.

I realized that the current setup is probably not very practical for me 
in the long run, though, as I now only have the integrated Intel GPU for 
my host to use, which will be quite a limitation for games and other 
graphics-intense applications. Therefore I am now considering getting a 
second card. You mentioned the GTX 750 (and Alex seems to have used this 
one successfully, too). Does it work well for you? If so, I might just 
try to get one of those. It should also be more or less on the same 
level of performance compared to the HD 7850 I am currently using.

On Wed, Apr 31, 2016 at 12:03 AM, Laszlo Ersek wrote:
> On 04/12/16 20:42, Stefan Seil wrote:
>> As far as I can tell, the other PCI device is the HDMI audio controller
>> of the graphics card. It is listed as a seperate PCI device and I
>> assigned it to the VM, too.
> I think that makes sense. I have the same situation with my GTX750. I
> believe those two functions belong to the same IOMMU domain, so they
> have to be assigned together anyway.
>
>> I haven't explicitly added any USB devices, although it seems like there
>> are some per default. I have linked the respective domain XMLs below.
> Hmm no, as far as device assignment goes, you have only two <hostdev>
> elements.
>
> Regarding USB devices, you have a few emulated controllers. As far as
> actual devices go, you seem to be using Spice USB redirection (see
> <redirdev>). I had to read up on that:
> <http://blog.wikichoon.com/2014/04/spice-usb-redirection-in-virt-manager.html>.
>
> I can't tell if you actually redirected some host-side USB devices. I
> think it's unlikely; Cole's blog post says "When connected to a VM's
> graphical display", and in your first domain XML, your only display is
> the assigned GPU (no <graphics> element).
>
>> I have played around with the configuration a bit and removed the rom
>> file for the HDMI audio controller of the graphics card (previously I
>> had given it the same rom file as the one I passed to the GPU)
> Aha! So that is why I saw two instances of AmdGop.efi.
>
> (I think there may be a bug in AmdGop.efi. A UEFI_DRIVER must cope with
> the situation when the PciIo protocol interface on the device handle is
> already open by another (or the same) driver.)
>
>> and now
>> the VM actually does not stop at the TianoCore logo anymore. The monitor
>> now displays the GRUB menu
> Great!
>
>> and turns black after choosing the first boot
>> entry. This is the new log now: http://pastebin.com/2dxBqP6h
> Yes, this one dispatches AmdGop.efi only once. I can also see
> shimx64.efi and grubx64.efi in the log.
>
>> I have also configured the VM with emulated graphics using QXL (just the
>> standard configuration you get with the setup wizard). If I try to boot
>> with the GPU passed through in addition to having the emulated graphics,
>> I can see it progressing further on the emulated display. It looks like
>> it starts booting the OS and then stops at this line:
>>
>> intel_rapl: no valid rapl domains found in package 0
>>
>> I don't see any of that on the monitor connected to the graphics card,
>> though.
> The point is, by eliminating the romfile for the HDMI audio controller,
> you managed to reach grub, and grub actually started your kernel. (The
> above intel_rapl message comes from "drivers/powercap/intel_rapl.c".)
>
>  From this point on, it is a guest kernel / guest GPU driver question.
> Since you do have functional grub, I recommend the following:
>
> * Edit your domain configuration so that it has an emulated serial
> console. Virtio-serial is *not* good enough. Run "virsh edit", and add
> the following:
>
>      <serial type='pty'>
>        <target port='0'/>
>      </serial>
>      <console type='pty'>
>        <target type='serial' port='0'/>
>      </console>
>
> ... Actually, I'm just noticing that your second XML paste, at
> <http://pastebin.com/Wm3E23nW>, already contains this XML fragment. Good.
>
> * Create the $HOME/.screenrc file, with the following content:
>
> logfile /tmp/screen.log
> logfile flush 1
>
> * Start the VM like this, from the shell:
>
>    # launch "screen" first
>    screen
>
>    # now press [Ctrl-A Shift-H]
>    # this will start logging terminal traffic to the logfile
>
>    # launch the guest
>    virsh start --console --paused xubuntu-clone
>
> * Wait until the line "Connected to domain xubuntu-clone" appears.
>
> * Now open the guest in virt-manager as well, and click the Pause button
> to unpause the guest.
>
> * As soon as the grub menu appears, choose to edit the kernel command
> line, under the first grub menu entry (i.e., the one that you would
> normally boot).
>
> If the "quiet" parameter is there, remove it.
>
> Independently, add the following three parameters:
>
>    console=tty console=ttyS0,115200 ignore_loglevel
>
> * Boot the edited command line.
>
> * When the guest becomes stuck again, force it off (using virt-manager),
> same as before. This will also make "virsh" return in your "screen"
> window, and then you can type "exit" there.
>
> * Email the /tmp/screen.log file, with the guest kernel's debug output,
> to whoever provided your guest kernel. :)
>
> Thanks
> Laszlo
>
>> These are the two domain XMLs now:
>> Without additional emulated graphics: http://pastebin.com/hBTdP6Sq
>> With additional emulated graphics: http://pastebin.com/Wm3E23nW
>