[vfio-users] Installing Catalyst prevents VM from booting.

[Konstantin Weber]

Hello everyone,

first time posting on a mailinglist, so I hope this works out.


A couple of days ago I had a simmilar issue with an amd gpu and windows
8.1. Maybe my notes could be helpful.


Background:
Hardware used:
Xeon E3-1246v3
MSI H97 Guard Pro
Asus R9 280 DC2T
24 GB Ram (2x8 GB, 2x4 GB)

Arch Linux with kenel 4.1.6-1

I have a working Windows 8.1 virtual machine with gpu passthrough with
an asus r9 280 dc2t.
This one was created with a <4.0 kernel (3.18 I think), pci-stub,
libvirt in qemu:\\\system over a year ago and has been running mostly
without problems. I could update the gpu driver and the ccc without
problems.
After I recently read at aw's blog that libvirt now supports passthrough
in user-session, i used vfio instead of pci-stub and could use the
virtual machine in user mode.


Actual issue:
For testing purposes I then wanted to create another virtual machine,
which then had the issue, that after installation of the gpu driver, the
vm would not boot unless the gpu was removed from it.
I used the same iso for installation, that I used for the working
virtual machine. After some attempts I even copied the installer of the
driver from one vm to the other, but it did not work.


Some of the steps I tried were:
- Installation of Catalyst 15.7.1 gave a bluescreen with
"system_thread_exception_not_handled (atikmdag.sys)".
- Attempted installation of the driver over the device manager resulted
in an immediate restart without installing the driver.
- Installation of the driver over the device manager in safe mode
worked, but a restart without safe mode resultet in permanent restarts
until the vm was forced shutdown.
- Copying the installed files (not the installer) from the windows/...
directories from the working one to the other one let me choose the
driver in the device manager, but also resulted in permanent restarts
until the gpu was removed from the vm.
- Starting the virtual machine over the command line without
virt-manager, installing the driver over device manager and rebooting
worked once and then resulted in restarts.

I tried for some days a couple of combinations of the above and some
suggestions I found through google, but nothing helped.


What finally did help was this:

Start a fresh installation of windows 8.1 in qemu without libvirt. It
contained only the minimum needed to start (cpu, ram ...; ovmf image;
installation iso; virtual disk; qxl vga; gpu for passthrough with audio).
During installation I disabled the windows setting for loading new
drivers. After the installation I didn't restart, update or change any
setting. The gpu had only the microsoft basic driver with a code 10 and
did not show anything.
I downloaded the 15.7.1 driver from amd, extracted it and stopped the
installation.
I opened the settings for resolution and disabled the output on the qxl
gpu, which left me with no output.
After that I killed and restarted the virtual machine, so the output was
only on the r9 280 and installed the driver manually over the device
manager. This worked fine and I had no bluescreen or restarts.
After the installation I was able to reboot the virtual machine without
problems and install the ccc including the hdmi driver. I could also
enable the qxl gpu, install new drivers for it and install virtio drivers.


My qemu start script now looks like this:

qemu-system-x86_64 -enable-kvm -m 4096 -cpu host,kvm=off -smp
4,sockets=1,cores=4,threads=1 \
-drive if=pflash,format=raw,readonly,file=/usr/share/ovmf/OVMF-pure-efi.fd \
-drive
if=pflash,format=raw,file=/home/MYUSERNAME/.config/libvirt/qemu/nvram/WinTest_ovmf.fd
\
-drive
file=/media/Daten/VirtuelleMaschinen/WinTest_ovmf.qcow2,if=virtio,format=qcow2
\
-vga qxl \
-device vfio-pci,host=01:00.0 -device vfio-pci,host=01:00.1