[vfio-users] Addition of machine flag vmport=off causes VM hang (100% CPU usage) when AMD driver loads

[Stewart Adam]

Hi all,

I've had BIOS Q35-based Windows 7 guest with a Radeon R9 270X GPU 
passthrough working very well with F21, but recently I upgraded to F23 and 
also needed to reformat my guest so I started from scratch.

Using virt-manager I created a new BIOS Q35 machine using the <emulator> 
script wrapper trick from the VFIO series part 5 [1], configured my 
pass-through devices and booted it up. VGA output worked well, but *only* 
using the Windows driver; using AMD's drivers (either Catalyst or Crimson) 
caused the VM to hang on a black screen with 100% guest CPU usage just after 
the Windows logo splashscreen at boot disappeared - I presume as soon as the 
AMD driver initialized. Safe mode or startup repair remained functional.

My old VM continued to work, so after lots and lots of config testing, I 
found that despite many differences I thought would matter between the 
configurations (machine type q35 1.5 vs 2.5, HyperV extensions, differing 
PCI IDs, toggling HPET), it was actually virt-manager's addition of the 
vmport=off machine flag that caused the hangs. After removing the flag, the 
guest now works well with AMD's driver installed. Interestingly, while 
troubleshooting the issue I sometimes needed to reboot the host to get 
things working again after a failed boot - i.e. my previous configuration, 
which worked upon clean boot, would also start to fail if I had tried 
booting a configuration with vmport=off first. With the flag removed, I've 
been able to reboot multiple times in a row with no issues.

Since the packages in F23 virt-preview now support the "ioh3420" 
pcie-root-port device, I'm very happy that it's now possible to use a fully 
native configuration without the need for manual QEMU args. Along with the 
removal of vmport=off and setting <emulator> (to patch in x-vga=on to the 
command line), things are working well.

Additional information:

Hardware
* CPU: Xeon E3-1225 v3
* GPU: AMD Radeon R9 270X
* Motherboard: Supermicro X10SAE

Software
* OS: Fedora 23 + virt-preview
* libvirt: 1.3.0-1.fc23.x86_64
* qemu: 2.5.0-1.fc23.x86_64
* kernel: 4.2.8-300.fc23.x86_64

My issues were very similar to this user's report, who I see also has 
vmport=off in the CLI flags:
https://www.redhat.com/archives/vfio-users/2015-November/msg00131.html

[1] http://vfio.blogspot.ca/2015/05/vfio-gpu-how-to-series-part-5-vga-mode.html

Regards,
Stewart