[vfio-users] VM doesn't boot if I use GPU passthrough
Ruben Felgenhauer
4felgenh at informatik.uni-hamburg.de
Tue Jan 26 17:35:59 UTC 2016
Hi, Ryan!
Installing an older Kernel is probably easier than you might think.
On Ubuntu you should be able to find out which kernels are in the repos
with apt-cache,
but I sadly don't know the params, so maybe take a look at the manpage.
And afterwards you should be able to install a specific version with
'apt-get install packagename=version'
On Debian there is simply http://snapshot.debian.org/package/linux/
which is how I downgraded from 4.3 to 4.1 on my Debian testing.
You can just download the deb files there and install them with dpkg.
Maybe if you search for a testing system that is similar to Ubuntu, you
could give that a try.
But keep in mind that this doesn't uninstall the old kernel, so you will
have a fallback.
You might need to select the right kernel at GRUB though.
Best regards,
Ruben
Am 26.01.2016 um 18:11 schrieb Will Marler:
> Well, you run Linux and you're experimenting with VGA passthrough ...
> you're resourceful! What about picking up a 16GB SSD for $15
> <http://www.amazon.com/Samsung-16GB-Solid-State-Drive/dp/B003YMJPE8/ref=sr_1_3?ie=UTF8&qid=1453827934&sr=8-3&keywords=16GB+SSD> and
> installing Arch (or Fedora, or Gentoo... whatever suits) side by side
> with Ubuntu? Presumably your VM can be launched either way without any
> configuration changes ... when you get tired/frustrated of the
> Arch/Fedora/Gentoo way you reboot back. If it works, you've found the
> answer, if it doesn't, you've improved your Linux-fu for not much
> (monetary) cost.
>
>
> On Tue, Jan 26, 2016 at 10:03 AM, Ryan Flagler <ryan.flagler at gmail.com
> <mailto:ryan.flagler at gmail.com>> wrote:
>
> Yea, that's just a major jump. Wish I had a dedicated test system
> to try more things. ;)
>
> On Tue, Jan 26, 2016 at 10:34 AM Will Marler <will at wmarler.com
> <mailto:will at wmarler.com>> wrote:
>
> Next up would be Kernel, it sounds like...
>
> On Tue, Jan 26, 2016 at 8:27 AM, Ryan Flagler
> <ryan.flagler at gmail.com <mailto:ryan.flagler at gmail.com>> wrote:
>
> Thanks for this info Will. Tried matching your
> qemu/libvirt versions and I still get the driver crashes.
> I'm not sure what else to try.
>
> On Mon, Jan 25, 2016 at 9:20 PM Will Marler
> <will at wmarler.com <mailto:will at wmarler.com>> wrote:
>
> Hey Ryan,
>
> Here are the answers to your questions:
>
> 20:06:27 will~% uname -a
> Linux haze 4.3.3-2-ARCH #1 SMP PREEMPT Wed Dec 23
> 20:09:18 CET 2015 x86_64 GNU/Linux
> 20:07:01 will~% pacman -Q | egrep '^linux|^libvirt|^qemu'
> libvirt 1.3.1-1
> libvirt-glib 0.2.2-1
> libvirt-python 1.3.1-1
> linux 4.3.3-2
> linux-api-headers 4.1.4-1
> linux-firmware 20151207.bbe4917-1
> qemu 2.4.1-2
>
> And here is the pastebin to my XML file:
> http://pastebin.com/nB3DPkEr
>
> As far as the guest drivers are concerned, they're the
> "GeForce Game Ready Driver" version 361.43.
>
> HTH!
>
> On Mon, Jan 25, 2016 at 10:12 AM, Ryan Flagler
> <ryan.flagler at gmail.com
> <mailto:ryan.flagler at gmail.com>> wrote:
>
> Thanks Will. Here is my info with the guest that
> crashes.
>
> Host OS Info
> ubuntu - 14.04.03
> kernel - 3.19.0-47
>
> virsh version
> Compiled against library: libvirt 1.2.18
> Using library: libvirt 1.2.18
> Using API: QEMU 1.2.18
> Running hypervisor: QEMU 2.5.0
>
> patches
> I did not manually apply any patches to Qemu.
> Built directly from source.
>
> Guest Info
> Windows 10
> nVidia Graphics Driver 361.43
>
> Guest Event Viewer Entry On Driver Crash
> Source - nvlddmkm
> Event ID - 14
> Info - \Device\Video3 CMDre 00000004 0000011c
> bad0011f 00000000 00d0011f
>
> Guest XML - Attached
>
>
> On Mon, Jan 25, 2016 at 10:18 AM Will Marler
> <will at wmarler.com <mailto:will at wmarler.com>> wrote:
>
> On Mon, Jan 25, 2016 at 9:07 AM, Ryan Flagler
> <ryan.flagler at gmail.com
> <mailto:ryan.flagler at gmail.com>> wrote:
>
> Will, could you tell us the following?
>
> What Linux distribution on host?
>
> Arch
>
> What kernel are you using on host?
> What libvirt version on host?
> What qemu version on host?
>
> Will have to check when I'm home from work &
> the kids are asnooze, but it's whatever's
> latest (and I'm not using the linux-vfio-lts
> kernel)
>
> What OS on guest?
>
> Windows 10.
>
> What nvidia graphics driver version on guest?
>
> Again, I'll have to check. But the latest or
> nearly latest.
>
> My machines gpu driver crashes constantly
> and I'm trying to narrow down why. Thanks!
>
> How frustrating : (. I'll also get a pastebin
> of my XML for you, in case that will help.
> I've been running "stable" since mid 2015. I
> use the quotes because some things tripped me
> up (guest machine can't "sleep," can only
> power on & power off; when host machine goes
> to sleep with guest running, on host wake-up
> the guest is non-responsive and 100% CPU).
>
> Will
>
>
> On Mon, Jan 25, 2016, 10:02 AM Will Marler
> <will at wmarler.com
> <mailto:will at wmarler.com>> wrote:
>
> This is discussed in
> http://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part-4-our-first.html.
> You have to do more than <kvm><hidden
> state='on'/></kvm>:
>
> "The GeForce card is nearly as easy,
> but we first need to work around some
> of the roadblocks Nvidia has put in
> place to prevent you from using the
> hardware you've purchased in the way
> that you desire (and by my reading
> conforms to the EULA for their
> software, but IANAL). For this step
> we again need to run virsh edit on the
> VM. Within the <features> section,
> remove everything between the <hyperv>
> tags, including the tags themselves.
> In their place add the following tags:
>
> <kvm>
> <hidden state='on'/>
> </kvm>
>
> Additionally, within the <clock> tag,
> find the timer named hypervclock,
> remove the line containing this tag
> completely. Save and exit the edit
> session."
>
> I can confirm it works, I've been
> getting a lot of mileage from my
> passed-through 750Ti lately since
> getting a Steam Link :-D.
>
> On Sun, Jan 24, 2016 at 7:32 AM, Ruben
> Felgenhauer
> <4felgenh at informatik.uni-hamburg.de
> <mailto:4felgenh at informatik.uni-hamburg.de>>
> wrote:
>
> Hi,
>
> finally I had time to this again.
> I tried out virt-manager and after
> a bit of playing around with it,
> it /somewhat/ worked:
>
> The machine is at least booting. I
> still have a standard vga card
> enabled in the virt-manager config
> window.
> After the machine has booted, I
> can see that the device gets
> recognized as 750ti.
> However, the gpu doesn't get used,
> because of 'Code 43'.
> Code 43 is a generic error, so any
> idea what it could mean in this case?
>
> Of course I added the <kvm><hidden
> state='on'/></kvm> lines at the
> associated position.
>
> Best regards,
> Ruben
>
>
> Am 18.01.2016 um 22:27 schrieb
> Will Marler:
>> I'm not sure what correct
>> command-line syntax is. Have you
>> tried using libvirt and
>> VirtManager to handle your VM
>> rather than command line, and
>> modifying the XML rather than the
>> command line? I think that's
>> generally the preferred method
>> these days (it's certainly easier
>> from my point of view, and the
>> way I got my 750 Ti to pass through).
>>
>> On Mon, Jan 18, 2016 at 11:04 AM,
>> Ruben Felgenhauer
>> <4felgenh at informatik.uni-hamburg.de
>> <mailto:4felgenh at informatik.uni-hamburg.de>>
>> wrote:
>>
>> Hi, Alex!
>>
>> Thanks for your reply!
>> My GPU indeed has a seperate
>> audio device located at 01:00.1.
>>
>> However, just adding -device
>> vfio-pci,host=01:00.1 doesn't
>> seem to do the trick.
>> Of course the corresponding
>> device is already blacklisted
>> and bound to vfio.
>>
>> The Debian Wiki entry about
>> VGA passthrough
>> (https://wiki.debian.org/VGAPassthrough)
>> mentions QEMU arguments like
>> "-device
>> vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on,romfile=...
>> -device
>> vfio-pci,host=01:00.1,bus=pcie.0"
>> which seems to address GPUs
>> with audio devices, but if I
>> try to do something similar,
>> the buses 'root' and 'pcie'
>> couldn't be found. Maybe I
>> missed something very important?
>>
>> On the same article, it says
>> that the "HDMI soundcard
>> [...] needs to be unbound
>> from its driver":
>> # echo '0000:01:00.1' | sudo
>> tee
>> /sys/bus/pci/devices/0000:01:00.1/driver/unbind
>> I figured the vfio-bind
>> script from the Arch Linux
>> Forum thread
>> (https://bbs.archlinux.org/viewtopic.php?id=162768)
>> would do exactly this thing,
>> so I didn't explicitly do so
>> for the audio device. Is that
>> okay?
>>
>> Best regards,
>> Ruben
>>
>>
>> Am 18.01.2016 um 08:31
>> schrieb Alexander Petrenz:
>>> Hi Ruben,
>>>
>>> I guess your 750ti also has
>>> some audio device. You
>>> should pass through this
>>> too. It should be something
>>> like 01:00.1. There are many
>>> command line examples you
>>> can find about that.
>>> Also I´m not quite sure, if
>>> you should remove the x-vga=on.
>>>
>>> Regards
>>> Alex
>>>
>>> On Sun, Jan 17, 2016 at
>>> 11:12 PM, Ruben Felgenhauer
>>> <4felgenh at informatik.uni-hamburg.de
>>> <mailto:4felgenh at informatik.uni-hamburg.de>>
>>> wrote:
>>>
>>> Hi,
>>>
>>> I am trying to pass my
>>> nVidia GTX 750ti to my
>>> QEMU guest.
>>>
>>> Problem is: After the
>>> QEMU monitor pops up,
>>> nothing happens. The
>>> GPU's output is dead,
>>> and the vm won't be
>>> accessible via SSH
>>> anymore, so it's very
>>> likely that the VM isn't
>>> booting up at all. Also,
>>> there are no error
>>> messages from QEMU on
>>> the console whatsoever
>>> which makes debugging it
>>> especially hard.
>>>
>>> This is how I start the
>>> vm with normal vga
>>> emulation:
>>> qemu-system-x86_64 -hda
>>> vm.ovl -boot c
>>> -enable-kvm -m 1024 -cpu
>>> host,kvm=off -smp
>>> cores=4,threads=2 -redir
>>> tcp:5022::22
>>> Everything runs fine in
>>> this case. To do the
>>> passthrough, I add this:
>>> -device
>>> vfio-pci,host=01:00.0,multifunction=on,x-vga=on
>>> -vga none
>>> This brings said
>>> problems with it. I also
>>> tried out multiple
>>> different combinations
>>> of -device's arguments
>>> or even adding a romfile
>>> for the GPU, but none of
>>> these steps changed
>>> anything at all.
>>>
>>> Obviously, I am using a
>>> BIOS installation and
>>> I'm well-aware with this
>>> bug:
>>> https://bugzilla.kernel.org/show_bug.cgi?id=107561,
>>> but neither using less
>>> RAM (as you can see I am
>>> using 1GB now) nor
>>> switching to an older
>>> Kernel changed anything
>>> about the problem. I
>>> have tried Kernel 4.1.0
>>> and 4.3.0.
>>>
>>> Host is Debian testing
>>> with QEMU 2.5.0.
>>> I tried both Debian and
>>> Windows 7 as a guest,
>>> but both are showing
>>> exactly the same behaviour.
>>> Mainboard is an ASUS
>>> Z87-PLUS. The 750ti is
>>> produced by ASUS aswell.
>>>
>>> Any idea how I could get
>>> passthrough running?
>>>
>>> _______________________________________________
>>> vfio-users mailing list
>>> vfio-users at redhat.com
>>> <mailto:vfio-users at redhat.com>
>>> https://www.redhat.com/mailman/listinfo/vfio-users
>>>
>>>
>>
>>
>> _______________________________________________
>> vfio-users mailing list
>> vfio-users at redhat.com
>> <mailto:vfio-users at redhat.com>
>> https://www.redhat.com/mailman/listinfo/vfio-users
>>
>>
>
>
> _______________________________________________
> vfio-users mailing list
> vfio-users at redhat.com
> <mailto:vfio-users at redhat.com>
> https://www.redhat.com/mailman/listinfo/vfio-users
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160126/3ccf5b9e/attachment.htm>
More information about the vfio-users
mailing list