[vfio-users] VM doesn't boot if I use GPU passthrough

[Will Marler]

Hey Ruben,

I think this might be problematic in your XML:

    <graphics type='spice' autoport='yes'>
      <image compression='off'/>
    </graphics>
    [...]
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x0'/>
    </video>
    [...]

I wouldn't play with the XML directly to remove them; rather, do it through
virt-manager. As a reference, here's <http://imgur.com/9722s0c> what my
virt-manager window looks like. My guess is you still have a spice & QXL
device.

I have an IGD and was completely uninterested in patching the kernel, so I
only ever tried UEFI/OVMF. I tried both Fedora and Arch, and ultimately
preferred Arch, so I installed the ovmf-git package to get Windows booting.
At the time I was working on getting my VM installed there was a bug in the
released OVMF code that would cause Windows to reboot during install. I'm
not sure if this is fixed or not (but you can search the forums and find me
asking about it, and find me getting pointed in the right direction).

Removing the Spice & QXL device might be all you need. But if you're
feeling adventurous, here's a fuller suggested solution:

 -- Create a new guest using an LVM partition. This allows you to create
snapshots (useful in one particular situation, see below).
 -- Spin up the VM guest using UEFI/OVMF. Get Windows 10 to boot & install.
(This might take a few tries; it took me 5 or 6 iirc)
 -- Install a VNC server in the guest, and test that it works.
 -- (snapshot & dump xml and ...) using virt-manager delete the
non-essential devices from the VM. Make the necessary tweaks to the XML
using virsh (deleting the hyperv stuff, kvm hidden state = on).
 -- restart the guest, and connect with VNC to make sure the guest is still
boots. If it doesn't, import the earlier XML & roll back the LVM snapshot &
try again (after creating a new LVM snapshot ofc!).
 -- Shut off the guest and configure the VGA devices in virt-manager to be
passed through
 -- Restart the guest and connect over VNC; the Windows Device Manager
should now see the graphics hardware. Install the device drivers.
 -- Reboot the guest and switch your monitor input and hold your breath!
 -- Once it works, don't forget to delete the LVM snapshot! (filling it up
would be a Bad Time).

Hope to hear of your success!

Will

On Sat, Jan 30, 2016 at 3:10 AM, Ruben Felgenhauer <
4felgenh at informatik.uni-hamburg.de> wrote:

> Hi,
>
> I tried your method and it definitely did something.
>
> When attaching the device as is, virsh tells me:
> error: internal error: Attempted double use of PCI slot 0000:00:02.0 (may
> need "multifunction='on'" for device on function 0)
> If I remove the second <address ... /> tag completely, virsh attaches the
> device successfully.
> Windows tells me, that the GPU has an error code 14 (that's new) and that
> I shall restart.
> After the restart, the GPU gives me Code 43 again.
>
> Best regards,
> Ruben
>
>
> Am 29.01.2016 um 01:56 schrieb Nicolas Roy-Renaud:
>
> Ok, try to remove your passthrough from your guest configuration (either
> using virsh of virt manager). That is : remove the actual gpu (PCI:1:0.0)
> but keep the associated sound card (PCI:1:0.1) in there so virsh knows it
> needs to bind with this VFIO group.
> From there, create a file (let's say ./GPU_DEVICE.xml) containing just the
> following :
>
>
>    1.     <hostdev mode='subsystem' type='pci' managed='yes'>
>    2.       <source>
>    3.         <address domain='0x0000' bus='0x01' slot='0x00' function=
>    '0x0'/>
>    4.       </source>
>    5.       <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
>    function='0x0'/>
>    6.     </hostdev>
>
>
> You'll be able to use this file to tell libvirt to append your GPU to
> guest's config at runtime, which somehow gets around the invalid ROM issue.
> Just run something like this :
>
> virsh start Win10Full && sleep 60 && virsh attach-device --live --file ./GPU_DEVICE.xml
>
> If I guessed right, windows should detect a new GPU and get the drivers in
> place once virsh is done mounting it. If that does work, you'll gave to run
> this same command every time you start your VM too, or at least until that
> specific bug is fixed. Hopefully that should get you some results so you
> can work your way from there.
>
> On 2016-01-28 17:03, Ryan Flagler wrote:
>
> I was going to recommend you use UEFI, which is why I was asking. I've
> personally had better luck getting things to pass through properly.
>
> Is your VM down when you try to cat the rom? The GPU needs to be unused by
> anything.
>
> I had the exact same symptoms on my Asus Strix 970, looks like a recurring
> issue with Asus cards. This happenned both when trying to start a VM with a
> managed passthrough and when attempting to dump the ROM from sysfs. I
> figured it's probably an issue with vfio-pci itself, and I still haven't
> fixed it wet, but the solution I posted above is my current workaround.
>
>
>
> _______________________________________________
> vfio-users mailing list
> vfio-users at redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160131/b758a138/attachment.htm>