[vfio-users] How does bios rom file work?

[Alex Williamson]

On Sat, 6 Jan 2018 15:27:00 -0800
Nick S <nick.kvmhv at gmail.com> wrote:

> I realized that I don't quite understand how does the ROM file that we can
> pass in QEMU command line work. I have three cards of the same model but
> coming from the different vendors. When I extract ROMs from these cards, I
> get different files. The card on the boot slot gives a much smaller file
> and other two cards give files of slightly different sizes:
> 
> -rw-rw-r--  1 nick nick 118784 Dec 14 21:53 vbiosrx480_03.rom
> -rw-rw-r--  1 nick nick  58880 Dec 14 21:53 vbiosrx480_04.rom
> -rw-rw-r--  1 nick nick 118272 Dec 14 19:57 vbiosrx480_06.rom
> 
> I am also seeing the following error in dmesg:
> [Sat Jan  6 09:26:58 2018] vfio-pci 0000:04:00.0: Invalid PCI ROM header
> signature: expecting 0xaa55, got 0xffff
> [Sat Jan  6 09:26:58 2018] vfio-pci 0000:04:00.0: Invalid PCI ROM header
> signature: expecting 0xaa55, got 0xffff

For the boot graphics device the kernel PCI code actually returns the
shadow ROM rather than reading it from the device's PCI option ROM.
This is also often also part of the reason that assigning a boot
graphics device might behave differently from a secondary device.  The
shadow ROM can be changed vs the on-device ROM.
 
> But other than that the card actually works without issues. So, the
> questions I have are:
> 
> 1) What is the significance of this ROM file and why is it needed?

It holds code that initializes the devices and provides the interface
for the BIOS to talk to it.  Effectively it's the BIOS-level device
driver for the device.

> 2) What would happen if I apply some changes to it? For example, I can edit
> it in Polaris bios editor. If I change the memory timings and use that
> changed file, will it take effect when this GPU is assigned to a VM or
> would the actual flashed bios take precedence?

ROMs have a checksum, so unless you fix that, the BIOS would reject the
ROM if modified (presumably a "bios editor" would do this for you, I'm
not familiar with Polaris).  The passed BIOS supersedes the onboard
BIOS, so quite likely the timings would be changed on VM boot.

> 3) What should I expect if I use bios from one maker of, say, Radeon 480 RX
> on the same model another maker (i.e. Sapphire vs AMD)?

IIRC, the subsystem vendor/device IDs are not used to match the ROM to
a device, so I'd expect the BIOS to try to use it.  Whether that's a
good thing or a bad thing for your hardware, I can't say.  The ROM
itself might care, but there are ways to masquerade the subsystem
vendor as well, see x-pci-sub-vendor-id, x-pci-sub-device-id options.
Obviously this is "at your own risk" territory.  Thanks,

Alex