<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Sat, Mar 11, 2017 at 6:24 AM, Patrick O'Callaghan <span dir="ltr"><<a href="mailto:poc@usb.ve" target="_blank">poc@usb.ve</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I'm a newbie when it comes to this stuff so would appreciate some<br> guidance. I've read Alex's long blog post and several other sources but<br> am still having trouble at the most basic level, i.e. masking my Nvidia<br> card.<br> <br> The system is an i7-3770 running Fedora 25 on an MSI board. VT-d is<br> enabled (as confirmed from dmesg). The second GPU is an Geforce GT630<br> (fairly low-end but I'll swap in a better one if I can get this all to<br> work). Basic info:<br> <br> $ lspci<br> 00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor DRAM Controller (rev 09)<br> 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09)<br></blockquote><div><br></div><div>There's no 00:02.0 VGA device here, which means IGD is disabled and only the NVIDIA card is being exposed to the OS. Assigning primary graphics to the guest is generally not recommended unless you know how to work around the labyrinth of issues that may result (ie. avoid if a newbie). Enable IGD and set it as the primary host graphics.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> 00:14.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller (rev 04)<br> 00:16.0 Communication controller: Intel Corporation 7 Series/C216 Chipset Family MEI Controller #1 (rev 04)<br> 00:1a.0 USB controller: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #2 (rev 04)<br> 00:1b.0 Audio device: Intel Corporation 7 Series/C216 Chipset Family High Definition Audio Controller (rev 04)<br> 00:1c.0 PCI bridge: Intel Corporation 7 Series/C216 Chipset Family PCI Express Root Port 1 (rev c4)<br> 00:1c.1 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 2 (rev c4)<br> 00:1d.0 USB controller: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #1 (rev 04)<br> 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a4)<br> 00:1f.0 ISA bridge: Intel Corporation B75 Express Chipset LPC Controller (rev 04)<br> 00:1f.2 SATA controller: Intel Corporation 7 Series/C210 Series Chipset Family 6-port SATA Controller [AHCI mode] (rev 04)<br> 00:1f.3 SMBus: Intel Corporation 7 Series/C216 Chipset Family SMBus Controller (rev 04)<br> 01:00.0 VGA compatible controller: NVIDIA Corporation GF108 [GeForce GT 630] (rev a1)<br> 01:00.1 Audio device: NVIDIA Corporation GF108 High Definition Audio Controller (rev a1)<br> 03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)<br> <br> $ find /sys/kernel/iommu_groups/ -type l<br> /sys/kernel/iommu_groups/7/<wbr>devices/0000:00:1c.1<br> /sys/kernel/iommu_groups/5/<wbr>devices/0000:00:1b.0<br> /sys/kernel/iommu_groups/3/<wbr>devices/0000:00:16.0<br> /sys/kernel/iommu_groups/11/<wbr>devices/0000:03:00.0<br> /sys/kernel/iommu_groups/1/<wbr>devices/0000:01:00.1<br> /sys/kernel/iommu_groups/1/<wbr>devices/0000:00:01.0<br> /sys/kernel/iommu_groups/1/<wbr>devices/0000:01:00.0<br> /sys/kernel/iommu_groups/8/<wbr>devices/0000:00:1d.0<br> /sys/kernel/iommu_groups/6/<wbr>devices/0000:00:1c.0<br> /sys/kernel/iommu_groups/4/<wbr>devices/0000:00:1a.0<br> /sys/kernel/iommu_groups/2/<wbr>devices/0000:00:14.0<br> /sys/kernel/iommu_groups/10/<wbr>devices/0000:00:1f.3<br> /sys/kernel/iommu_groups/10/<wbr>devices/0000:00:1f.2<br> /sys/kernel/iommu_groups/10/<wbr>devices/0000:00:1f.0<br> /sys/kernel/iommu_groups/0/<wbr>devices/0000:00:00.0<br> /sys/kernel/iommu_groups/9/<wbr>devices/0000:00:1e.0<br> <br> So IOMMU group 1 contains my Nvidia GPU and audio, plus the PCIe root<br> bridge. <br> <br> $ cat /etc/modprobe.d/local.conf <br> options vfio-pci ids=10de:ffffffff:ffffffff:<wbr>ffffffff:00030000:ffff00ff,<wbr>10de:ffffffff:ffffffff:<wbr>ffffffff:00040300:ffffffff<br> <br> (this is just cut-and-paste from Alex's blog. I've no idea what any of<br> the magic hex numbers mean except for the 10de of course).<br> <br> $ cat /etc/dracut.conf.d/local.conf<br> add_drivers+="vfio vfio_iommu_type1 vfio_pci vfio_virqfd"<br> <br> My boot line is:<br> <br> GRUB_CMDLINE_LINUX="rd.<wbr>blacklist=nvidia rd.blacklist=nouveau intel_iommu=on iommu=pt rd.driver.pre=vfio-pci vconsole.font=latarcyrheb-<wbr>sun16 $([ -x /usr/sbin/rhcrashkernel-param ] && /usr/sbin/rhcrashkernel-param || :) rhgb quiet resume=U<br> UID=1431e6d2-531e-46cd-8633-<wbr>1cf878c6b2a1" audit=0<br> <br> (Note that I have the Nvidia proprietary driver so need to mask that as<br> well as Nouveau).<br></blockquote><div><br></div><div>Or just uninstall it.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> My monitor is connected to HDMI on the internal GPU and DVI on the<br> Nvidia card.<br></blockquote><div><br></div><div>IGD is disabled in BIOS according to the info presented.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> However on rebooting I always have the Nvidia module loaded, and don't<br> have vfio:<br> $ lsmod|egrep -i nvidia\|vfio<br> nvidia_drm 53248 1<br> nvidia_modeset 806912 8 nvidia_drm<br> nvidia 12267520 175 nvidia_modeset<br> drm_kms_helper 151552 1 nvidia_drm<br> drm 339968 4 nvidia_drm,drm_kms_helper<br></blockquote><div><br></div><div>What do you actually see for the commandline in /proc/cmdline?</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <br> And of course, the monitor is only active on the DVI port and not the HDMI.<br> <br> Any suggestions on what I might try?<br></blockquote><div><br></div><div>Enable IGD in the BIOS, set it as primary graphics, uninstall nvidia driver in the host, verify the kernel commandline is what you think via proc and that your initramfs is being rebuilt with the drivers you want, use lsinitrd to check. </div></div></div></div>