[vfio-users] vfio PCI passthrough NVIDIA K420 GPU problems
Daniel Pocock
daniel at pocock.pro
Thu Feb 25 12:58:01 UTC 2016
On 24/02/16 23:18, Alex Williamson wrote:
> On Wed, Feb 24, 2016 at 2:45 PM, Daniel Pocock <daniel at pocock.pro
> <mailto:daniel at pocock.pro>> wrote:
>
>
>
> On 24/02/16 21:58, Alex Williamson wrote:
> >
> >
> > On Wed, Feb 24, 2016 at 1:26 PM, Bronek Kozicki <brok at spamcop.net <mailto:brok at spamcop.net>
> > <mailto:brok at spamcop.net <mailto:brok at spamcop.net>>> wrote:
> >
> > On 24/02/2016 15:41, Daniel Pocock wrote:
> >
> >
> >
> > Hi,
> >
> > I'm trying to use PCI passthrough to give an NVIDIA GPU to
> a VM with
> > qemu / KVM. I've summarized my environment below and the
> error
> > I get is
> > near the bottom. Any help would be appreciated.
> >
> > There are a few guides I've been referring to already:
> > https://wiki.debian.org/VGAPassthrough
> >
> https://www.pugetsystems.com/labs/articles/Multiheaded-NVIDIA-Gaming-using-Ubuntu-14-04-KVM-585/
> > https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF
> > https://bbs.archlinux.org/viewtopic.php?id=162768
> > http://www.linux-kvm.org/page/VGA_device_assignment
> >
> >
> >
> > Hi Daniel
> >
> >
> > I'm successfully passing through two Quadro M5000 (to two
> instances
> > of Windows 10), looking at
> >
> http://us.download.nvidia.com/Windows/Quadro_Certified/361.91/361.91-win10-quadro-grid-release-notes.pdf
> > (page 9) it is not obvious that this would work for either
> your K420
> > or mine M5000. One gotcha - when starting Windows I do not see
> boot
> > screen at all, only Windows logon scren after nVidia drivers had
> > loaded. Explanation is in Alex's email sent to this list on
> 6th Feb
> > 2016, subject "No boot screen on Quadro M5000?" - basically passed
> > through Quadro cards are meant to be secondary only (I use
> them as
> > only card, though). However this small quirk aside, this works for
> > me very well.
> >
> >
> >
> > Unfortunately K420 falls into the gap of unsupported and unknown
> between
> > GeForce, which are clearly unsupported, but we know how to make them
> > work, and Quardo K2000+, which are supported. Typically Quadro cards
> > don't work as primary like GeForce do, and the NVIDIA driver will
> pull a
> > Code 43 on them, so it probably requires some combination of secondary
> > plus hiding KVM & Hyper-v (or changing hyper-v vendor info) to make it
> > work. I'll note though that K420 is a GK107GL, as is the K2000, so
> > x-pci-device-id might be an interesting experiment. And of
> course, the
> > most important guide was missed from the list, the one at
> > vfio.blogspot.com <http://vfio.blogspot.com>
> <http://vfio.blogspot.com> ;) Thanks,
> >
>
>
> Thanks for all the replies about this
>
> What do you mean by primary vs secondary, is that documented somewhere?
> Do you mean I need some other card to be primary in the VM?
>
> I have two cards I can use for this VM, the K420 and a Quadro 2000.
>
> The Quadro 2000 is mentioned favorably in the Xen guide[1], am I likely
> to have more luck with that than the K420? The Quadro 2000 is a HP card
> and HP provides a BIOS download[2] too.
>
>
> In a secondary configuration, an emulated graphics is the primary
> graphics and is used for pre-boot, early boot, and install for the VM.
> Once accelerated drivers are installed and enabled, the primary graphics
> is generally disabled (automatically or manually depending on the guest
> OS). In a primary graphics configuration, the assigned GPU is the
> primary, typically exclusive, graphics for the VM. The 2000 is probably
> more likely to work because NVIDIA is potentially not explicitly
> preventing it from working like they do for the K420 and GeForce cards.
> It is a Fermi based card though, so it's not something we would
> actually claim as supported by NVIDIA. Add it to the VM just as you
> would any other PCI assigned device, keeping the emulated graphics
> configuration. Thanks,
>
OK, thanks for all the feedback, I've tried the following and the screen
is still blank (Code 12 in Device Manager, but at one stage it was Code
43 with the K420):
- updated from qemu in Debian jessie (2.1) to the version from
jessie-backports (2.5)
- using pci-stub on the host GPU (the K2200), X is not running at all on
the host
- adding the ignore_msrs=1 option to the kvm module, so it now uses:
options kvm allow_unsafe_assigned_interrupts=1 ignore_msrs=1
- adding rombar=0 to the -device config
- with the K420, I've tried adding
x-pci-vendor-id=0x10DE,x-pci-device-id=0x0FFE to the -device config and
Device Manager now thinks it is a K2000. It still says Code 12 though,
once it showed Code 43.
- I'm using emulated (qxl, also tried std) as primary display and
currently aiming to make either GPU (Quadro 2000 or K420) work as
secondary. I've also tried -nographic a few times with each.
- if I have the qemu monitor open and I try to supply the ROM to the
device, I see things like this at startup:
(qemu) dma: command 78 not supported
dma: command 78 not supported
dma: command 40 not supported
dma: command 40 not supported
- I can now get the OVMF shell on the emulated graphics (but not on
either GPU) and use the "pci" command to see the list of PCI devices,
the GPU is included in the list. OVMF won't boot the Windows 7 DVD or
the partition though, whenever I try to boot from either (using the boot
menu) it just fails and goes back to the OVMF shell, the error appears
too quickly for me to see what is happening.
Here are some relevant bits of the shell script I'm using to try
different permutations of qemu command lines:
#VGA="-nographic"
#VGA="-vga none"
#VGA="-vga std -vnc 0:15"
VGA="-vga qxl -vnc 0:15"
#ROMFILE="/root/NVIDIA-Quadro-2000-HP-70.06.3F.00.04_12320500.rom"
#ROMFILE="/root/nvidia-quadro-2000.rom"
#ROMFILE="/root/nvidia-k420.rom"
#ROMARG=",romfile=${ROMFILE}"
ROMARG=",rombar=0"
#ROMARG=",rombar=0,romfile=${ROMFILE}"
SIM_DEVICE=",x-pci-vendor-id=0x10DE,x-pci-device-id=0x0FFE"
# also consider x-pci-sub-vendor-id x-pci-sub-device-id?
#VGA_PASSTHROUGH="-device
ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
#-device
vfio-pci,host=42:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on${ROMARG}${SIM_DEVICE}
\
#-device vfio-pci,host=42:00.1,bus=root.1,addr=00.1"
VGA_PASSTHROUGH="-device
vfio-pci,host=42:00.0,bus=pcie.0,addr=09.0,multifunction=on,x-vga=on${ROMARG}${SIM_DEVICE}
\
-device vfio-pci,host=42:00.1,bus=pcie.0,addr=09.1"
Can anybody suggest anything else that I should try?
I've attached a list of cards listed by one of the application vendors,
are any of these easier to get going in a passthrough configuration? I
don't mind changing the card if an alternative will be easier to
configure or more reliable in the long term.
Regards,
Daniel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: recommended-cards.csv
Type: text/comma-separated-values
Size: 3260 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160225/f2705e17/attachment.bin>
More information about the vfio-users
mailing list