[vfio-users] "Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff" when attempting to dump PCI ROM

[Auger Eric]

Hi Nicolas,

On 3/10/19 11:06 PM, Nicolas Roy-Renaud wrote:
> I've seen a lot of people before reccomand VFIO newcomers to flash their
> GPU if they couldn't get their passthrough working right before, and
> since I know how potentially risky and avoidable this sort of procedure
> is (since QEMU lets you just pass your own ROM to the VM to be used
> instead), and while attempting to go through the steps myself
> <http://vfio.blogspot.com/2014/08/does-my-graphics-card-rom-support-efi.html>
> (even though I've had a working VFIO setup for years), got something
> unexpected.
> 
> Attempting to dump the ROM from my guest card _/freshly /_/_after a
> reboot_/ results in the following error message :
> 
> cat: '/sys/bus/pci/devices/0000:07:00.0/rom': Input/output error
> 
> Accompanied by the following like in dmesg :
> 
> [ 1734.316429] vfio-pci 0000:07:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff

Last time I got this error was when a readw was attempted in
pci_map_rom/pci_get_rom_size while the Device Control memory space
access bit was disabled in PCI cfg space. At the time you attempt to
dump the rom, the guest may have disabled this bit?

Thanks

Eric
> 
> This isn't a recent thing, I started investigating this several months
> ago
> <https://www.reddit.com/r/VFIO/comments/971umf/is_everyone_else_getting_invalid_pci_rom_header/>
> and it turned out that I wasn't alone: Everyone on the VFIO discord chat
> who actually tried this got the same error message.
> 
> The interesting part is that launching and shutting down a VM using that
> device clears the issue and it then becomes possible to dump the ROM as
> expected. I've known a number of issues with VFIO and ROMs can be
> resolved either like this or by attaching the guest GPU on an already
> live VM (for reasons I admittedly don't quite understand myself), but
> since getting a ROM dump can be an important part of setting up the VM
> that will receive the device in the first place, I don't know if this
> could be considered an option.
> 
> I figured this had something to do with the host BIOS tainting the ROM
> on boot, which is a well known thing on boot GPUs (even though the one
> I'm attempting to dump isn't), and that passing the tainted GPU to a VM
> then causes the device to power cycle and clears that, somehow? With
> that in mind, I've attempted a number of things, but neither resetting
> the PCI port the device is attached to
> <https://unix.stackexchange.com/a/474378>, nor doing function-reset on
> the device, nor going in and out of sleep mode on the guest seems to
> resolve the issue. I've also tried using pci-stub instead of vfio-pci to
> see if this wasn't a driver issue, but no success here either.
> 
> Is there anyone with better technical knowledge of VFIO who could help
> with this?
> 
> - Nicolas
> 
> 
> _______________________________________________
> vfio-users mailing list
> vfio-users at redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>