[vfio-users] IGD pass through, black screen, block on running option rom in seabios

Thu Aug 23 07:57:11 UTC 2018

Yesterday I tried IGD with Ubuntu 18.04 but failed:
https://www.redhat.com/archives/vfio-users/2018-August/msg00026.html

Today I use the same hardware enviroment, reinstall OS to CentOS 7.3
without GUI, upgrade kernel and qemu version:

CPU: i3-4010U, support vt-d.

[root at test ~]# lspci | grep VGA
00:02.0 VGA compatible controller: Intel Corporation Haswell-ULT Integrated
Graphics Controller (rev 09)
(Only one IGD with vga and hdmi interface)

[root at test ~]# uname -r
4.18.3-1.el7.centos.x86_64

[root at test ~]# /usr/libexec/qemu-kvm --version
QEMU emulator version 2.10.0(qemu-kvm-ev-2.10.0-21.el7_5.4.1)
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers

Then I blacklist i915, forbid *fb:

[root at test ~]# cat /etc/default/grub | grep GRUB_CMDLINE_LINUX
GRUB_CMDLINE_LINUX="crashkernel=auto rhgb quiet intel_iommu=on iommu=pt
video=vesafb:off vga=normal nofb nomodeset i915.modeset=0
nouveau.modeset=0 rd.driver.blacklist=nouveau,i915 plymouth.ignore-udev"

[root at test ~]# cat /etc/modprobe.d/blacklist-i915.conf
install i915 /bin/true

After rebooted:
[root at test ~]# cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-4.18.3-1.el7.centos.x86_64
root=UUID=78feccee-2f23-4ab6-b386-2b51d1664da7 ro crashkernel=auto rhgb
quiet intel_iommu=on iommu=pt video=vesafb:off vga=normal nofb nomodeset
i915.modeset=0 nouveau.modeset=0 rd.driver.blacklist=nouveau,i915
plymouth.ignore-udev

[root at test ~]# cat /proc/iomem | grep fb
  01c031d1-02463fbf : Kernel data
f8000000-fbffffff : PCI MMCONFIG 0000 [bus 00-3f]
  f8000000-fbffffff : Reserved
    f8000000-fbffffff : pnp 00:05

[root at test ~]# lsmod | grep i915
... nothing ...

Now I begin to load vfio modules.

[root at test ~]# modprobe vfio
[root at test ~]# modprobe vfio-pci

[root at test ~]# echo "vfio-pci" >
/sys/bus/pci/devices/0000:00:02.0/driver_override
[root at test ~]# ./vfio-pci-bind.sh 0000:00:02.0
(script ref:
https://github.com/andre-richter/vfio-pci-bind/blob/master/vfio-pci-bind.sh)

[root at test ~]# lspci -v -s 00:02.0
00:02.0 VGA compatible controller: Intel Corporation Haswell-ULT Integrated
Graphics Controller (rev 09) (prog-if 00 [VGA controller])
    Subsystem: Intel Corporation Haswell-ULT Integrated Graphics Controller
    Flags: fast devsel, IRQ 16
    Memory at a7c00000 (64-bit, non-prefetchable) [size=4M]
    Memory at b0000000 (64-bit, prefetchable) [size=256M]
    I/O ports at 4000 [size=64]
    [virtual] Expansion ROM at 000c0000 [disabled] [size=128K]
    Capabilities: [90] MSI: Enable- Count=1/1 Maskable- 64bit-
    Capabilities: [d0] Power Management version 2
    Capabilities: [a4] PCI Advanced Features
    Kernel driver in use: vfio-pci
    Kernel modules: i915

Now virsh create a win7 vm, xml file is:

<domain type='kvm' id='3' xmlns:qemu='
http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>windows_7_ultimate_with_sp1_x64</name>
  <uuid>2fcccb90-f622-481a-8819-367c642c0a09</uuid>
  <memory unit='KiB'>2097152</memory>
  <currentMemory unit='KiB'>2097152</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-rhel7.5.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
    </hyperv>
  </features>
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>Haswell-noTSX</model>
    <feature policy='require' name='vme'/>
    <feature policy='require' name='f16c'/>
    <feature policy='require' name='rdrand'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='arat'/>
    <feature policy='require' name='xsaveopt'/>
    <feature policy='require' name='abm'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/home/windows_7_ultimate_with_sp1_x64.qcow2'/>
      <backingStore/>
      <target dev='hda' bus='ide'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x0'/>
    </hostdev>
  </devices>

    <qemu:commandline>
     <qemu:arg value='-bios'/>
      <qemu:arg value='/usr/share/seabios/bios.bin'/>
      <qemu:arg value='-chardev'/>
      <qemu:arg value='file,id=seabios,path=/tmp/bios.log'/>
      <qemu:arg value='-device'/>
      <qemu:arg value='isa-debugcon,iobase=0x402,chardev=seabios'/>
    </qemu:commandline>
</domain>

After created, the vm is running, monitor screen turn to black then no
output.
The end of seabios log is:

...
Scan for VGA option rom
Running option rom at c000:0003

Any problem with VGA rom? In the BIOS I have already enabled CSM and the
option rom of video is set to legacy.

Ref to http://lists.gnu.org/archive/html/qemu-discuss/2018-04/msg00052.html
and https://github.com/awilliam/rom-parser

I dump the vga rom:

[root at test rom]# echo 1 > /sys/devices/pci0000:00/0000:00:02.0/rom
[root at test rom]# cat /sys/devices/pci0000:00/0000:00:02.0/rom > vbios.dump
[root at test rom]# echo 0 > /sys/devices/pci0000:00/0000:00:02.0/rom

[root at test rom]# ./rom-parser/rom-parser vbios.dump
Valid ROM signature found @0h, PCIR offset 40h
    PCIR: type 0 (x86 PC-AT), vendor: 8086, device: 0406, class: 030000
    PCIR: revision 3, vendor revision: 0
    Last image

[root at test rom]# lspci -nns 00:02.0
00:02.0 VGA compatible controller [0300]: Intel Corporation Haswell-ULT
Integrated Graphics Controller [8086:0a16] (rev 09)

The device code in the rom is diffrent from vga device? So I modify it with
rom-fixer command:

[root at test rom]# ./rom-parser/rom-fixer ./vbios.dump
Valid ROM signature found @0h, PCIR offset 40h
    PCIR: type 0 (x86 PC-AT), vendor: 8086, device: 0406, class: 030000
    PCIR: revision 3, vendor revision: 0

Modify vendor ID 8086? (y/n): y
New vendor ID: 8086
Overwrite vendor ID with 8086? (y/n): y
Modify device ID 0406? (y/n): y
New device ID: 0a16
Overwrite device ID with 0a16? (y/n): y
    Last image
ROM checksum is invalid, fix? (y/n): y

Then I edit the vm xml, and romfile args to hostdev:
      <rom file='/path/to/vbios.dump'/>

Then destroy and create the vm again, still black screen ...

Is there any more methods to debug？
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20180823/82deaa44/attachment.htm>