[vfio-users] "Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff" when attempting to dump PCI ROM

[Nicolas Roy-Renaud]

I've seen a lot of people before reccomand VFIO newcomers to flash their 
GPU if they couldn't get their passthrough working right before, and 
since I know how potentially risky and avoidable this sort of procedure 
is (since QEMU lets you just pass your own ROM to the VM to be used 
instead), and while attempting to go through the steps myself 
<http://vfio.blogspot.com/2014/08/does-my-graphics-card-rom-support-efi.html> 
(even though I've had a working VFIO setup for years), got something 
unexpected.

Attempting to dump the ROM from my guest card _/freshly /_/_after a 
reboot_/ results in the following error message :

cat: '/sys/bus/pci/devices/0000:07:00.0/rom': Input/output error

Accompanied by the following like in dmesg :

[ 1734.316429] vfio-pci 0000:07:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff

This isn't a recent thing, I started investigating this several months 
ago 
<https://www.reddit.com/r/VFIO/comments/971umf/is_everyone_else_getting_invalid_pci_rom_header/> 
and it turned out that I wasn't alone: Everyone on the VFIO discord chat 
who actually tried this got the same error message.

The interesting part is that launching and shutting down a VM using that 
device clears the issue and it then becomes possible to dump the ROM as 
expected. I've known a number of issues with VFIO and ROMs can be 
resolved either like this or by attaching the guest GPU on an already 
live VM (for reasons I admittedly don't quite understand myself), but 
since getting a ROM dump can be an important part of setting up the VM 
that will receive the device in the first place, I don't know if this 
could be considered an option.

I figured this had something to do with the host BIOS tainting the ROM 
on boot, which is a well known thing on boot GPUs (even though the one 
I'm attempting to dump isn't), and that passing the tainted GPU to a VM 
then causes the device to power cycle and clears that, somehow? With 
that in mind, I've attempted a number of things, but neither resetting 
the PCI port the device is attached to 
<https://unix.stackexchange.com/a/474378>, nor doing function-reset on 
the device, nor going in and out of sleep mode on the guest seems to 
resolve the issue. I've also tried using pci-stub instead of vfio-pci to 
see if this wasn't a driver issue, but no success here either.

Is there anyone with better technical knowledge of VFIO who could help 
with this?

- Nicolas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20190310/5ade9f64/attachment.htm>