[vfio-users] Nvidia Code 43 on Fedora 24 host and Windows 10 guest.

[Alex Williamson]

On Tue, 4 Oct 2016 19:19:09 -0600
Alex Williamson <alex.williamson at redhat.com> wrote:

> On Tue, 4 Oct 2016 21:40:17 +0200
> Dawid Osuchowski <tulphoon at openmailbox.org> wrote:
> 
> > I've rebooted the machine, the lspci output was the same without
> > launching anything prior to it.
> > 
> > If by OVMF boot splash screen you mean the "TianoCore" screen before the
> > OS is starting, then yes i'm able to see it.
> > 
> > I've put the ROM file in the /usr/share/qemu directory and run
> > restorecon on it (and changed the path in virsh) as you've suggested and
> > booted up the VM, still doesn't work. I've changed slot from 0x02 to
> > 0x08, had to reinstall the drivers, because system just freaked out and
> > it still doesn't work afterwards.
> > 
> > The AUX power cord is connected to the GPU. I've been using this card
> > before as a main GPU for this machine and then decided to give
> > passthrough a go.
> > 
> > Driver version is 372.90.  
> 
> I happen to have an EVGA GTX660, so I did a clean FC24 host install,
> followed my blog using your email and XML to see what paths you took,
> and it all worked like it's supposed to.  But my GTX660 has UEFI
> support in the onboard ROM, so I didn't need to specify a stand-alone
> ROM.  Trying to align more to your configuration I dumped the ROM using
> GPU-z in the guest (I wasn't able to get it from the Linux host
> directly).  When I specify this ROM for the VM, the ROM dump from my
> own card, I now get Code43 in the guest.  So, I don't have answers
> yet, I need to see if the ROM differs from a bare metal GPU-z dump as
> well as from what QEMU reads from the card.  Barring differences there,
> see if I can find any differences in handling of the ROM for native vs
> side-load.  Thanks,

Well, I managed to fix my ROM.  If I look at the ROM with xxd (from vim
package), it starts off like this:

00000000: 4e56 4749 d501 1000 6004 0000 4238 6026  NVGI....`...B8`&
00000010: 08e2 0000 0006 0000 0210 1082 ffff ff7f  ................
00000020: 0000 0080 0e10 1082 ffff ff73 0000 008c  ...........s....
00000030: 3610 1002 ffff ffff 0000 0000 1c24 0280  6............$..
00000040: 8100 0000 5281 0802 ffff ffff 0000 0000  ....R...........
00000050: 4024 1200 0000 0000 c026 1200 0000 0000  @$.......&......
00000060: c024 1280 ffff ffff 4026 1280 0020 0000  .$......@&... ..
00000070: 1c84 0880 0004 3600 a880 0800 0300 0000  ......6.........

Typically a PCI option ROM starts with 55aa, so this already looks a
little funky.  I notice the non-UEFI ROM at techpowerup for this card
starts the same way.  If I scroll down a bit, I find this:

00000600: 55aa 7beb 4b37 3430 30e9 4c19 77cc 5649  U.{.K7400.L.w.VI
00000610: 4445 4f20 0d00 0000 9001 2c1b 0000 4942  DEO ......,...IB
00000620: 4d20 5647 4120 436f 6d70 6174 6962 6c65  M VGA Compatible
00000630: 0100 0000 c000 b652 3132 2f31 342f 3132  .......R12/14/12
00000640: 0000 0000 0000 0000 0010 4000 c1f4 0000  .......... at .....
00000650: e9a4 2b00 4238 6026 ffff ff7f 0000 0000  ..+.B8`&........
00000660: ffff ff73 0000 008c b553 a5db e9fa 45e9  ...s.....S....E.
00000670: 0146 504d 4944 6c00 6f00 0000 00a0 00b0  .FPMIDl.o.......
00000680: 00b8 00c0 0033 474b 3130 3620 5032 3033  .....3GK106 P203
00000690: 3020 534b 5520 3020 5647 4120 4249 4f53  0 SKU 0 VGA BIOS

This is what the head of the option ROM should look like.  So to make a
working ROM, I simply ran:

dd if=GK106.rom of=GK106-new.rom bs=1 skip=1536

Where 1536 is the decimal conversion of 0x600, which is the offset of
the 55aa PCI option ROM header.

Unfortunately this may be a different problem than you have since you
indicated the following ROM dump output:

output from EfiRom -d /opt/strip.rom
Image 1 -- Offset 0x0
  ROM header contents
    Signature              0xAA55
...
Image 2 -- Offset 0xF600
  ROM header contents
    Signature              0xAA55

So you have aa55 signatures at offsets 0x0 and 0xf600, which matches my
new ROM after it's been "fixed", maybe your ROM has a different issue.
If you want to send me your ROM file, I'll compare to mine and see
whether it works with my card (not sure whether it should or not given
different vendors).  Thanks,

Alex