[vfio-users] IGD passthrough: No Seabios output; PTE read access not set
Manuel Ullmann
ullman.alias at posteo.de
Sat Dec 10 01:07:43 UTC 2016
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hi,
trying to pass through an Haswell IGD into a Windows 10 qemu VM. For
hardware specs, see [3]. Don’t got any display output so far, although
Grub screen is reset and replaced by no signal. No vfio-pci errors are left.
In this setup no drivers exist except i915. There is no discrete
graphics card. Efifb and simplefb are not built by the kernel. i915 is
removed from root filesystem and exists only in initrd, where my init
script prevents loading of the i915 module, if booting into vm runlevel
is passed via grub edit. See [5] for hopefully relevant kernel options.
I don’t get any output after grub and IGD is boot GPU. When the IGD is
grabbed by VM, I see shortly artifacts (similar to unbinding in running
OS) and the signal turns off. From HDD activity and Windows log I can
tell that the VM is booting.
Solved the BAR 2 allocating error due to BOOTFB allocated IGD memory
with the kernel module from [4]. As noted above, it is quite impossible
in this setup, that any Linux kernel driver grabs the IGD. My assumption
for it happening anyway would be, that the allocation is caused by the
video driver of the UEFI firmware itself. If you have a look into the
simplefb help, you could notice, that it does no device initialization
at all. It writes to an existing framebuffer of a preinitialized device.
A backing for that assumption would be the behavior of Win8+ systems in
pure (non-CSM) UEFI mode: Those keep displaying the boot splash (Intel
mainboard logo for me) and render the loading circle below. I assume
that works similar to simplefb. Probably this allocation can be avoided,
if the host system is installed in CSM mode, so that the successful pass
through reports could be explained. In any case, the gpu ram release
would be a nice optional quirk addition to vfio-pci.
The kernel log has a few page fault messages indicating PTE read access
not being set [2]. Maybe those are the culprits, although Alex
considered them harmless in earlier posts, if they don’t keep popping up
after the binding, which they do not. I also let the VM run for a few
hours, hoping Windows would eventually find a driver, but it seems to be
a general passthrough issue. Anyway, if I let it run, I always got a kvm
error like this:
kvm [13836]: vcpu0, guest rIP: 0xfffff80c096c6067 unhandled rdmsr: 0x641
kernel: 4.9.0-rc6
qemu: 2.7.0-r7
See version history for patching information (security patches mostly):
https://gitweb.gentoo.org/repo/gentoo.git/log/app-emulation/qemu?showmsg=1
I’m not sure, what else I could provide.
Attaching Qemu command line, grepped kernel log, hardware specs, vga
quirk, grepped kernel config, qemu output and my qemu openrc service.
If you need the complete log or kernel config, please let me know. I
don’t know, what to do from here, please give me a hand.
Best regards,
Manuel
[1]: Qemu command line
qemu-system-x86_64 -cpu host\
-usb\
-machine type=pc,accel=kvm\
-enable-kvm\
-vga none\
-nographic\
-display none\
-nodefconfig\
-display none\
-boot once=d\
-bios /usr/share/qemu/bios.bin\
-m 2G\
-drive
file=/dev/genhd/ReactOS,cache=none,if=virtio,format=raw,index=0\
-drive
file=/mnt/platz/Betriebssystemabbilder/virtio-win-0.1.118.iso,media=cdrom,index=2\
-smb /mnt/platz/QemuShare\
# headphone DAC
-usbdevice host:1019:0011\
# mouse
-usbdevice host:046d:c246\
-device
vfio-pci,host=00:02.0,addr=02.0,bus=pci.0,x-vga=on,romfile=/mnt/platz/Bugs/intelBios/intelHaswell.rom\
-netdev user,id=vmnic -device virtio-net,netdev=vmnic
&>/var/log/qemuvm.log
[2]: Kernel log
grep -aiE '((DMAR)|(kvm)|(drm)|(Command line)|(iommu)|(vfio))'
/var/log/messages
Dec 10 00:57:21 sonnengebleicht [ 0.000000] Command line:
BOOT_IMAGE=/vmlinuz-4.9.0-rc6 root=/dev/mapper/gensd-gentoo ro quiet
splash intel_iommu=on video=efifb:off,vesafb:off,simplef
b:off splash=verbose softlevel=qemuvm
Dec 10 00:57:21 sonnengebleicht [ 0.000000] ACPI: DMAR
0x00000000DB3F2498 0000B8 (v01 INTEL DH87RL 0000014A INTL 00000001)
Dec 10 00:57:21 sonnengebleicht [ 0.000000] Kernel command line:
BOOT_IMAGE=/vmlinuz-4.9.0-rc6 root=/dev/mapper/gensd-gentoo ro quiet
splash intel_iommu=on video=efifb:off,vesafb:off,
simplefb:off splash=verbose softlevel=qemuvm
Dec 10 00:57:21 sonnengebleicht [ 0.000000] DMAR: IOMMU enabled
Dec 10 00:57:21 sonnengebleicht [ 0.013952] DMAR: Host address width 39
Dec 10 00:57:21 sonnengebleicht [ 0.013953] DMAR: DRHD base:
0x000000fed90000 flags: 0x0
Dec 10 00:57:21 sonnengebleicht [ 0.013959] DMAR: dmar0:
reg_base_addr fed90000 ver 1:0 cap c0000020660462 ecap f0101a
Dec 10 00:57:21 sonnengebleicht [ 0.013959] DMAR: DRHD base:
0x000000fed91000 flags: 0x1
Dec 10 00:57:21 sonnengebleicht [ 0.013962] DMAR: dmar1:
reg_base_addr fed91000 ver 1:0 cap d2008020660462 ecap f010da
Dec 10 00:57:21 sonnengebleicht [ 0.013962] DMAR: RMRR base:
0x000000dbeb8000 end: 0x000000dbec6fff
Dec 10 00:57:21 sonnengebleicht [ 0.013964] DMAR: RMRR base:
0x000000dd000000 end: 0x000000df1fffff
Dec 10 00:57:21 sonnengebleicht [ 0.013966] DMAR-IR: IOAPIC id 8
under DRHD base 0xfed91000 IOMMU 1
Dec 10 00:57:21 sonnengebleicht [ 0.013967] DMAR-IR: HPET id 0 under
DRHD base 0xfed91000
Dec 10 00:57:21 sonnengebleicht [ 0.014209] DMAR-IR: Enabled IRQ
remapping in xapic mode
Dec 10 00:57:21 sonnengebleicht [ 0.388693] DMAR: No ATSR found
Dec 10 00:57:21 sonnengebleicht [ 0.388933] DMAR: dmar0: Using Queued
invalidation
Dec 10 00:57:21 sonnengebleicht [ 0.388938] DMAR: dmar1: Using Queued
invalidation
Dec 10 00:57:21 sonnengebleicht [ 0.388982] DMAR: Setting RMRR:
Dec 10 00:57:21 sonnengebleicht [ 0.389006] DMAR: Setting identity
map for device 0000:00:02.0 [0xdd000000 - 0xdf1fffff]
Dec 10 00:57:21 sonnengebleicht [ 0.389210] DMAR: Setting identity
map for device 0000:00:14.0 [0xdbeb8000 - 0xdbec6fff]
Dec 10 00:57:21 sonnengebleicht [ 0.389237] DMAR: Setting identity
map for device 0000:00:1a.0 [0xdbeb8000 - 0xdbec6fff]
Dec 10 00:57:21 sonnengebleicht [ 0.389263] DMAR: Setting identity
map for device 0000:00:1d.0 [0xdbeb8000 - 0xdbec6fff]
Dec 10 00:57:21 sonnengebleicht [ 0.389275] DMAR: Prepare 0-16MiB
unity mapping for LPC
Dec 10 00:57:21 sonnengebleicht [ 0.389292] DMAR: Setting identity
map for device 0000:00:1f.0 [0x0 - 0xffffff]
Dec 10 00:57:21 sonnengebleicht [ 0.389410] DMAR: Intel(R)
Virtualization Technology for Directed I/O
Dec 10 00:57:21 sonnengebleicht [ 0.389452] iommu: Adding device
0000:00:00.0 to group 0
Dec 10 00:57:21 sonnengebleicht [ 0.389458] iommu: Adding device
0000:00:02.0 to group 1
Dec 10 00:57:21 sonnengebleicht [ 0.389463] iommu: Adding device
0000:00:03.0 to group 2
Dec 10 00:57:21 sonnengebleicht [ 0.389468] iommu: Adding device
0000:00:14.0 to group 3
Dec 10 00:57:21 sonnengebleicht [ 0.389474] iommu: Adding device
0000:00:16.0 to group 4
Dec 10 00:57:21 sonnengebleicht [ 0.389479] iommu: Adding device
0000:00:19.0 to group 5
Dec 10 00:57:21 sonnengebleicht [ 0.389485] iommu: Adding device
0000:00:1a.0 to group 6
Dec 10 00:57:21 sonnengebleicht [ 0.389490] iommu: Adding device
0000:00:1b.0 to group 7
Dec 10 00:57:21 sonnengebleicht [ 0.389494] iommu: Adding device
0000:00:1d.0 to group 8
Dec 10 00:57:21 sonnengebleicht [ 0.389502] iommu: Adding device
0000:00:1f.0 to group 9
Dec 10 00:57:21 sonnengebleicht [ 0.389507] iommu: Adding device
0000:00:1f.2 to group 9
Dec 10 00:57:21 sonnengebleicht [ 0.389511] iommu: Adding device
0000:00:1f.3 to group 9
Dec 10 00:57:21 sonnengebleicht [ 0.395792] [drm] Initialized
Dec 10 00:57:21 sonnengebleicht [ 0.575518] VFIO - User Level
meta-driver version: 0.3
Dec 10 00:57:21 sonnengebleicht [ 0.707783] ata1.00: supports DRM
functions and may not be fully accessible
Dec 10 00:57:21 sonnengebleicht [ 0.717345] ata1.00: supports DRM
functions and may not be fully accessible
Dec 10 00:57:21 sonnengebleicht [ 0.808124] vfio_pci: add
[8086:0412[ffff:ffff]] class 0x000000/00000000
Dec 10 00:57:22 sonnengebleicht kernel: DMAR: DRHD: handling fault
status reg 3
Dec 10 00:57:22 sonnengebleicht kernel: DMAR: [DMA Write] Request device
[00:02.0] fault addr dd5c0000 [fault reason 05] PTE Write access is not set
Dec 10 00:57:22 sonnengebleicht kernel: DMAR: DRHD: handling fault
status reg 3
Dec 10 00:57:22 sonnengebleicht kernel: DMAR: [DMA Read] Request device
[00:02.0] fault addr dd5c0000 [fault reason 06] PTE Read access is not set
Dec 10 00:57:22 sonnengebleicht kernel: DMAR: DRHD: handling fault
status reg 2
alternating once
Dec 10 00:57:22 sonnengebleicht kernel: DMAR: [DMA Read] Request device
[00:02.0] fault addr dd5c2000 [fault reason 06] PTE Read access is not set
Dec 10 00:57:22 sonnengebleicht kernel: DMAR: DRHD: handling fault
status reg 3
Dec 10 00:57:22 sonnengebleicht kernel: DMAR: [DMA Read] Request device
[00:02.0] fault addr dd5c3000 [fault reason 06] PTE Read access is not set
Dec 10 00:57:22 sonnengebleicht kernel: DMAR: DRHD: handling fault
status reg 2
Dec 10 00:57:22 sonnengebleicht kernel: DMAR: [DMA Read] Request device
[00:02.0] fault addr dd5c3000 [fault reason 06] PTE Read access is not set
repeated 3 times
Dec 10 00:57:22 sonnengebleicht kernel: DMAR: [DMA Read] Request device
[00:02.0] fault addr dd5c4000 [fault reason 06] PTE Read access is not set
Dec 10 00:57:23 sonnengebleicht kernel: kvm: zapping shadow pages for
mmio generation wraparound
Dec 10 00:57:24 sonnengebleicht kernel: kvm: zapping shadow pages for
mmio generation wraparound
[3]: Hardware specs
CPU: Intel Haswell i5-4570
http://ark.intel.com/products/75043/Intel-Core-i5-4570-Processor-6M-Cache-up-to-3_60-GHz
Mainboard: Intel DH87RL
Ark does not contain a Vt-d row for the DH87RL mainboard unfortunately.
But it is available and activated in Bios as well as Virtualization
Technology. Found a thread with Intel employee confirmation.
https://communities.intel.com/thread/45010
Monitor is attached to DVI.
[4]: vgaquirk for BAR reservation failure
https://lists.gnu.org/archive/html/qemu-devel/2016-07/msg02469.html
Module Makefile based on example 1 of kernel doc modules.txt :
ifneq ($(KERNELRELEASE),)
# kbuild part of makefile
obj-m := forcefully-remove-bootfb.o
else
# normal makefile
KDIR ?= /lib/modules/`uname -r`/build
default:
INSTALL_MOD_DIR=vgaquirk
$(MAKE) -C $(KDIR) M=$$PWD
endif
[5] Kernel config:
grep -E '^CONFIG.*((FB)|(DRM)|(VFIO)|(KVM))' .config
CONFIG_X86_SYSFB=y
CONFIG_DRM=y
CONFIG_DRM_MIPI_DSI=y
CONFIG_DRM_KMS_HELPER=y
CONFIG_DRM_KMS_FB_HELPER=y
CONFIG_DRM_FBDEV_EMULATION=y
CONFIG_DRM_I915=m
CONFIG_DRM_I915_USERPTR=y
CONFIG_DRM_PANEL=y
CONFIG_DRM_BRIDGE=y
CONFIG_FB=y
CONFIG_FB_CMDLINE=y
CONFIG_FB_NOTIFY=y
CONFIG_FB_CFB_FILLRECT=y
CONFIG_FB_CFB_COPYAREA=y
CONFIG_FB_CFB_IMAGEBLIT=y
CONFIG_FB_SYS_FILLRECT=y
CONFIG_FB_SYS_COPYAREA=y
CONFIG_FB_SYS_IMAGEBLIT=y
CONFIG_FB_SYS_FOPS=y
CONFIG_FB_DEFERRED_IO=y
CONFIG_VFIO_IOMMU_TYPE1=y
CONFIG_VFIO_VIRQFD=m
CONFIG_VFIO=y
CONFIG_VFIO_PCI=m
CONFIG_VFIO_PCI_VGA=y
CONFIG_VFIO_PCI_MMAP=y
CONFIG_VFIO_PCI_INTX=y
CONFIG_VFIO_PCI_IGD=y
CONFIG_HAVE_KVM=y
CONFIG_HAVE_KVM_IRQCHIP=y
CONFIG_HAVE_KVM_IRQFD=y
CONFIG_HAVE_KVM_IRQ_ROUTING=y
CONFIG_HAVE_KVM_EVENTFD=y
CONFIG_KVM_MMIO=y
CONFIG_KVM_ASYNC_PF=y
CONFIG_HAVE_KVM_MSI=y
CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT=y
CONFIG_KVM_VFIO=y
CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT=y
CONFIG_KVM_COMPAT=y
CONFIG_HAVE_KVM_IRQ_BYPASS=y
CONFIG_KVM=y
CONFIG_KVM_INTEL=y
[6]: qemu output
qemu-system-x86_64: -smb /mnt/platz/QemuShare: The -smb option is
deprecated. Please use '-netdev user,smb=...' instead.
qemu-system-x86_64: Warning: speed mismatch trying to attach usb device
"FOSTEX USB AUDIO HP-A4" (high speed) to bus "usb-bus.0", port "1" (full
speed)
repeated 2 times
qemu-system-x86_64: terminating on signal 15 from pid 3442
[7]: qemu openrc service
#!/sbin/openrc-run
# Copyright 1999-2016 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Id$
depend() {
need localmount
use net netmount nfsmount
}
checkconfig() {
if ! [ -f /root/startQemu.sh ]; then
eerror "qemu start script does not exist"
return 1
fi
return 0
}
start() {
checkconfig || return 1
ebegin "Starting qemu VM"
start-stop-daemon --background --exec /root/startQemu.sh
return=$?
pidof qemu-system-x86_64 > /var/run/qemuVM.pid
eend $return
}
stop() {
ebegin "Stopping qemuVM"
start-stop-daemon --stop -p /var/run/qemuVM.pid
eend $?
}
-----BEGIN PGP SIGNATURE-----
iQQcBAEBCAAGBQJYS1VbAAoJELgK9xQKkpX61dEgAI7x2Cdv7Slq4EdLBZah95v5
YQfPY77ETd1OUdv5JphYmMncyggBYgQheqQSU8T5G6/ZpNlD1qh29+oGDiQThP8s
7toTPvKte6NXqKtsvGDDTbUWbUVyN2mDy/o/PlXkxm8ZbMy8s/XDHgcKLlbDq9nr
LpUaMpuMQja4BqGhF+sbqB4iocHJXOIZNN+VOJC8CCmfSFbwof1oXkfJn52HKR+L
Vjy1ZczSrrior52lfFAyxs//X2x9l9xUE8KZYwzeyksG5xUN7ZbJYNYK4w/KfLih
T6W56xYo6gbsT7+XA97xrDn0+OiHb/Ciji2sXVUuLVasscrPFDMv5ISSmBdlOXhg
wORGXkUG3SHfHNmZMqgc/6fYBkm00y8azXGNVo1eOwOdioM8gZZTYTpeNXYzrk22
E/swEljz6rH8xbhwprSBOZM/1CzalFDlAsVXgLh4lXgKrnFmp+9d7SbSEa3eHRqy
/lr5dsOUQOED0mbwYqSmkYv0Zq6lSE6LYi9QICjYFoQEgiKTYPsDFGb/ji1d0bfS
JzD2hz9WF9CU9OaatAbn9b8jRnXVjlaTGn5ax1nQVuggMvts5HPP+Nt/16OS5Oce
Xthr5+fRxk58v3njOx2Ec8Sfl4EQd0LZwBhDK/mVeYQRMCQB+gvEiLfmPZRBz50s
0LCQTyFsFQKIbf7Zp/5hQvBQe9+JIsT353KOUX3ZvgRhEV5ixyBWB6g6u9fXuKs4
M26ud05Pxz9QskZUDZIeqfGW0sb4eE9lXPovJMSTwCFcNfenYNLiaysgR2eKuytK
LDaaK2ebTKdgWltRjRFgMkQyDWS//fGhhRj6TKALeQjJokJXQUhepWDtxrEA1vQz
8ZWH0e8/EjLPERMSHJgnFNavxzFQDatY55a4TcwcXjwzwmb7sgySiJZdfST5UH9f
dS+nMOoU66BtIdpkoGdh6LxM5L1gEFKckpHSctM65gi85d8icL4lr2WNCJAmHlSj
szf9DMfTbB/w7pLXRpE3hm0UyP3ArFNlZ97A/2mjOT0/hx1tOHSFm7j8YKewli08
9mgxCdalWejoQ8As8wM7+c+25fu2BxrIBWSXOrlZPY3AnoX2upQRaCPiA03JOhOP
BbwnGzk3SSn6ltPmJf/g258x05NAxRi8dvoUamTZavpcP5Oj4djVd9N8sOBLNLd7
dgQk4kiv6l9eLjXhf8xBX/RXRiNAKqH5cJ6Ltoxcfsb5dR4RqyTNieLO0bQ2H91z
mEGyp41Z6C86aI0sShq0HGEjPe40KdlKqZbuki11ibFTdkXFGk5kIIiIp2Y0TX+X
Ln8os1v8dAaH4HBvQZr8Cb5aGNC4DKluTdRlVsXp4NygMIjh9LdDv03x0tcb2ho=
=RZ72
-----END PGP SIGNATURE-----
More information about the vfio-users
mailing list