[vfio-users] Newbie steps
Patrick O'Callaghan
poc at usb.ve
Sun Mar 12 19:35:54 UTC 2017
On Sun, 2017-03-12 at 12:53 -0600, Alex Williamson wrote:
> Enable IGD in the BIOS, set it as primary graphics, uninstall nvidia driver
> in the host, verify the kernel commandline is what you think via proc and
> that your initramfs is being rebuilt with the drivers you want, use
> lsinitrd to check.
OK, did that and things are improving. I should mention that I had
previously tried unplugging the Nvidia card and the system booted with
the internal GPU, which lead me to think it was going to be enabled,
but you were right, it wasn't in the BIOS.
I haven't uninstalled the Nvidia drivers (yet) because of caution, but
I have now rebooted and get:
# lspci -nn
00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor DRAM Controller [8086:0150] (rev 09)
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0151] (rev 09)
00:02.0 VGA compatible controller [0300]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor Graphics Controller [8086:0162] (rev 09)
00:14.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller [8086:1e31] (rev 04)
00:16.0 Communication controller [0780]: Intel Corporation 7 Series/C216 Chipset Family MEI Controller #1 [8086:1e3a] (rev 04)
00:1a.0 USB controller [0c03]: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #2 [8086:1e2d] (rev 04)
00:1b.0 Audio device [0403]: Intel Corporation 7 Series/C216 Chipset Family High Definition Audio Controller [8086:1e20] (rev 04)
00:1c.0 PCI bridge [0604]: Intel Corporation 7 Series/C216 Chipset Family PCI Express Root Port 1 [8086:1e10] (rev c4)
00:1c.1 PCI bridge [0604]: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 2 [8086:1e12] (rev c4)
00:1d.0 USB controller [0c03]: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #1 [8086:1e26] (rev 04)
00:1e.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev a4)
00:1f.0 ISA bridge [0601]: Intel Corporation B75 Express Chipset LPC Controller [8086:1e49] (rev 04)
00:1f.2 SATA controller [0106]: Intel Corporation 7 Series/C210 Series Chipset Family 6-port SATA Controller [AHCI mode] [8086:1e02] (rev 04)
00:1f.3 SMBus [0c05]: Intel Corporation 7 Series/C216 Chipset Family SMBus Controller [8086:1e22] (rev 04)
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF108 [GeForce GT 630] [10de:0f00] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation GF108 High Definition Audio Controller [10de:0bea] (rev a1)
03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 06)
As you can see the 3rd line (00:02.0 ...) is new. And the IOMMU list
shows it in Group 2 so no interference with the Nvidia one.
$ lsmod|grep vfio
vfio_pci 45056 0
irqbypass 16384 2 kvm,vfio_pci
vfio_virqfd 16384 1 vfio_pci
vfio_iommu_type1 20480 0
vfio 28672 2 vfio_iommu_type1,vfio_pci
$ lsmod|grep nvidia
$ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-4.9.13-201.fc25.x86_64 root=UUID=6a9a800c-309b-4b69-b269-884ad495e7b9 ro modprobe.blacklist=nvidia modprobe.blacklist=nouveau rd.blacklist=nvidia rd.blacklist=nouveau intel_iommu=on iommu=pt rd.driver.pre=vfio-pci vconsole.font=latarcyrheb-sun16 rhgb quiet resume=UUID=1431e6d2-531e-46cd-8633-1cf878c6b2a1
Sure enough the monitor is running off the HDMI cable while the DVI
output is dead, as it should be. My only quibble is a spew of error
messages in dmesg:
# dmesg|grep DMAR
[ 0.000000] ACPI: DMAR 0x00000000BD5B7800 0000B8 (v01 INTEL SNB 00000001 INTL 00000001)
[ 0.000000] DMAR: IOMMU enabled
[ 0.032824] DMAR: Host address width 36
[ 0.032825] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[ 0.032831] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap c0000020e60262 ecap f0101a
[ 0.032831] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[ 0.032834] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap c9008020660262 ecap f0105a
[ 0.032834] DMAR: RMRR base: 0x000000bd422000 end: 0x000000bd42ffff
[ 0.032835] DMAR: RMRR base: 0x000000bf800000 end: 0x000000cf9fffff
[ 0.032837] DMAR-IR: IOAPIC id 2 under DRHD base 0xfed91000 IOMMU 1
[ 0.032837] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[ 0.032838] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.033076] DMAR-IR: Enabled IRQ remapping in x2apic mode
[ 0.720048] DMAR: No ATSR found
[ 0.720497] DMAR: dmar0: Using Queued invalidation
[ 0.720501] DMAR: dmar1: Using Queued invalidation
[ 0.784439] DMAR: Software identity mapping for device 0000:00:00.0
[ 0.784440] DMAR: Software identity mapping for device 0000:00:01.0
[ 0.784444] DMAR: Software identity mapping for device 0000:00:02.0
[ 0.784445] DMAR: Software identity mapping for device 0000:00:14.0
[ 0.784445] DMAR: Software identity mapping for device 0000:00:16.0
[ 0.784446] DMAR: Software identity mapping for device 0000:00:1a.0
[ 0.784447] DMAR: Software identity mapping for device 0000:00:1b.0
[ 0.784448] DMAR: Software identity mapping for device 0000:00:1c.0
[ 0.784448] DMAR: Software identity mapping for device 0000:00:1c.1
[ 0.784449] DMAR: Software identity mapping for device 0000:00:1d.0
[ 0.784450] DMAR: Software identity mapping for device 0000:00:1f.0
[ 0.784451] DMAR: Software identity mapping for device 0000:00:1f.2
[ 0.784451] DMAR: Software identity mapping for device 0000:00:1f.3
[ 0.784455] DMAR: Software identity mapping for device 0000:01:00.0
[ 0.784456] DMAR: Software identity mapping for device 0000:01:00.1
[ 0.784460] DMAR: Software identity mapping for device 0000:03:00.0
[ 0.784461] DMAR: Setting RMRR:
[ 0.784462] DMAR: Setting identity map for device 0000:00:02.0 [0xbf800000 - 0xcf9fffff]
[ 0.785486] DMAR: Setting identity map for device 0000:00:14.0 [0xbd422000 - 0xbd42ffff]
[ 0.785488] DMAR: Setting identity map for device 0000:00:1a.0 [0xbd422000 - 0xbd42ffff]
[ 0.785489] DMAR: Setting identity map for device 0000:00:1d.0 [0xbd422000 - 0xbd42ffff]
[ 0.785491] DMAR: Prepare 0-16MiB unity mapping for LPC
[ 0.785491] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[ 0.785710] DMAR: Intel(R) Virtualization Technology for Directed I/O
[ 0.866268] DMAR: 32bit 0000:00:1a.0 uses non-identity mapping
[ 0.866294] DMAR: Setting identity map for device 0000:00:1a.0 [0xbd422000 - 0xbd42ffff]
[ 0.877136] DMAR: 32bit 0000:00:1d.0 uses non-identity mapping
[ 0.877162] DMAR: Setting identity map for device 0000:00:1d.0 [0xbd422000 - 0xbd42ffff]
[ 1.602494] [drm] DMAR active, disabling use of stolen memory
[ 1.622730] DMAR: DRHD: handling fault status reg 2
[ 1.622739] DMAR: [DMA Write] Request device [00:02.0] fault addr 7300000000 [fault reason 05] PTE Write access is not set
[ 1.729748] DMAR: DRHD: handling fault status reg 2
[ 1.729756] DMAR: [DMA Write] Request device [00:02.0] fault addr 7300000000 [fault reason 05] PTE Write access is not set
[ 1.940292] DMAR: DRHD: handling fault status reg 2
[ 1.940335] DMAR: [DMA Write] Request device [00:02.0] fault addr 7300000000 [fault reason 05] PTE Write access is not set
[ 2.746359] DMAR: DRHD: handling fault status reg 2
[ 2.746389] DMAR: [DMA Write] Request device [00:02.0] fault addr 7300000000 [fault reason 05] PTE Write access is not set
Not sure if that's a BIOS thing I need to look at or what.
However, definite progress so many thanks. No doubt I'll have other
questions as time goes on ...
poc
More information about the vfio-users
mailing list