<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Sat, Sep 12, 2015 at 11:04 AM, Janusz <span dir="ltr"><<a href="mailto:januszmk6@gmail.com" target="_blank">januszmk6@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Hello,<br> <br> I have a question about allowing unsafe interrupts, what exactly is<br> happening when we allow it? why its unsafe?<br></blockquote><div><br></div><div>On a standard PC, Message Signaled Interrupts (MSI) are triggered via a DMA write by the device to a special address range. In the early revisions of VT-d, the IOMMU did not protect this range, which allowed devices to effectively spoof interrupts from other devices and potentially run attacks against the host using DMA writes to this interrupt block. Later versions of VT-d introduced the interrupt remapping feature which, among other things, protects this range so that a device can only signal the interrupts programmed for it.</div><div><br></div><div>Another thing that interrupt remapping does is provide a translation between IOAPIC and X2APIC interrupt domains on the system. Pretty much all new Intel processors support X2APIC, which necessitates interrupt remapping support. So, I'd be really surprised if your new skylake system doesn't have interrupt remapping. You should really only need to enable the unsafe interrupts option if you try to assign the device, it fails, and dmesg tells you that you need it. And only then if you trust your guests not to be malicious to the host.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> I am asking because I am not able to passthrough my gpu without allowing<br> unsafe interrupts, and have problems in running my virtual machine as it<br> sometimes rebooting its self before its able to run windows. I know that<br> there is also problem with using iGPU and not uefi bios, but its also<br> happening with OVMF and passing uefi rom for my gpu (except that with<br> OVMF sometimes I also get very high load). without passing VGA,<br> everyting works fine. Is it possible that its because those unsafe<br> interrupts?<br> My hardware: i7 6700k, MSI z170a M7, Sapphire R9 290<br></blockquote><div><br></div><div>Congratulations, you're the first person to show up with a Skylake system, welcome to trailblazing a new Intel platform. When you say you're not able to assign the GPU without the unsafe interrupts option, does that mean when you run the VM QEMU exits and dmesg reports:</div><div><br></div><div>"No interrupt remapping support. Use the module param "allow_unsafe_interrupts" to enable VFIO IOMMU support on this platform"</div><div><br></div><div>That would be pretty epic if Intel dropped interrupt remapping support on Skylake. Or perhaps you just mean that you have something closer to working with unsafe interrupts. If the hardware supports interrupt remapping, then the unsafe option makes absolutely no difference. It's only meant to allow an otherwise un-allowed scenario by making the user opt-in to the security risk.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> those are my options for non-uefi:<br> -enable-kvm -m 10000 -cpu host -smp 8,cores=4,threads=2,sockets=1<br> -device<br> ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1<br> -device vfio-pci,host=01:00.0,multifunction=on,x-vga=on<br> -device vfio-pci,host=01:00.1<br></blockquote><div><br></div><div>Looks like you're adding an ioh3420 for absolutely no reason. Same with the multifunction option.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> and for uefi:<br> <br> -drive if=pflash,format=raw,readonly,file=/home/janusz/uefi/OVMF_CODE.fd<br> -drive if=pflash,format=raw,file=/home/janusz/uefi/OVMF_VARS.fd<br> -enable-kvm -m 10000 -cpu host -smp 8,cores=4,threads=2,sockets=1<br> -device<br> vfio-pci,host=01:00.0,romfile=/home/janusz/uefi/uefi-vga.bin,multifunction=on<br> -device vfio-pci,host=01:00.1<br></blockquote><div><br></div><div>Useless multifunction option again. Why does this one specify a ROM file? Does the on-card ROM not support UEFI?</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> + options with assigning /dev/sdb and two usb devices<br> <br> I am using ovmf build recently from master, kernel 4.2.0 and qemu-2.4.50<br> <br> <br> My iommu groups:<br> <br> /sys/kernel/iommu_groups/0/devices/0000:00:00.0<br></blockquote><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> /sys/kernel/iommu_groups/1/devices/0000:00:01.0<br> /sys/kernel/iommu_groups/1/devices/0000:01:00.0<br> /sys/kernel/iommu_groups/1/devices/0000:01:00.1<br> </blockquote><div><br></div><div>So this ought to be the group we care about.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">/sys/kernel/iommu_groups/2/devices/0000:00:02.0<br> /sys/kernel/iommu_groups/3/devices/0000:00:08.0<br> /sys/kernel/iommu_groups/4/devices/0000:00:14.0<br> /sys/kernel/iommu_groups/4/devices/0000:00:14.2<br> /sys/kernel/iommu_groups/5/devices/0000:00:15.0<br> /sys/kernel/iommu_groups/5/devices/0000:00:15.1<br> /sys/kernel/iommu_groups/6/devices/0000:00:16.0<br> /sys/kernel/iommu_groups/7/devices/0000:00:17.0<br></blockquote><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> /sys/kernel/iommu_groups/8/devices/0000:00:1c.0<br> /sys/kernel/iommu_groups/8/devices/0000:00:1c.2<br> /sys/kernel/iommu_groups/8/devices/0000:00:1c.7<br> /sys/kernel/iommu_groups/8/devices/0000:02:00.0<br> /sys/kernel/iommu_groups/8/devices/0000:03:00.0<br> /sys/kernel/iommu_groups/8/devices/0000:04:00.0<br></blockquote><div><br></div><div>Lovely, all the PCH root ports are grouped together, people looking for hardware please take note.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> /sys/kernel/iommu_groups/9/devices/0000:00:1e.0<br> /sys/kernel/iommu_groups/10/devices/0000:00:1f.0<br> /sys/kernel/iommu_groups/10/devices/0000:00:1f.2<br> /sys/kernel/iommu_groups/10/devices/0000:00:1f.3<br> /sys/kernel/iommu_groups/10/devices/0000:00:1f.4<br> <br> and lspci:<br> <br> 00:00.0 Host bridge: Intel Corporation Sky Lake Host Bridge/DRAM<br> Registers (rev 07)<br> </blockquote><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">00:01.0 PCI bridge: Intel Corporation Sky Lake PCIe Controller (x16)<br> (rev 07)<br> </blockquote><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">00:02.0 VGA compatible controller: Intel Corporation Sky Lake Integrated<br> Graphics (rev 06)<br> </blockquote><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">00:08.0 System peripheral: Intel Corporation Sky Lake Gaussian Mixture Model<br> </blockquote><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">00:14.0 USB controller: Intel Corporation Sunrise Point-H USB 3.0 xHCI<br> Controller (rev 31)<br> 00:14.2 Signal processing controller: Intel Corporation Sunrise Point-H<br> Thermal subsystem (rev 31)<br> </blockquote><div><br></div><div>Don't you like how they've put the USB3 controller on a multifunction device with some random thermal management device, without isolation of course. Wonder what chaos assigning that to a VM would cause.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">00:15.0 Signal processing controller: Intel Corporation Sunrise Point-H<br> LPSS I2C Controller #0 (rev 31)<br> 00:15.1 Signal processing controller: Intel Corporation Sunrise Point-H<br> LPSS I2C Controller #1 (rev 31)<br> </blockquote><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">00:16.0 Communication controller: Intel Corporation Sunrise Point-H CSME<br> HECI #1 (rev 31)<br> </blockquote><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">00:17.0 SATA controller: Intel Corporation Device a102 (rev 31)<br> </blockquote><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">00:1c.0 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root<br> Port #1 (rev f1)<br> 00:1c.2 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root<br> Port #3 (rev f1)<br> 00:1c.7 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root<br> Port #8 (rev f1)<br> 00:1e.0 Signal processing controller: Intel Corporation Sunrise Point-H<br> LPSS UART #0 (rev 31)<br> </blockquote><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">00:1f.0 ISA bridge: Intel Corporation Sunrise Point-H LPC Controller<br> (rev 31)<br> 00:1f.2 Memory controller: Intel Corporation Sunrise Point-H PMC (rev 31)<br> 00:1f.3 Audio device: Intel Corporation Sunrise Point-H HD Audio (rev 31)<br> 00:1f.4 SMBus: Intel Corporation Sunrise Point-H SMBus (rev 31)<br></blockquote><div><br></div><div>Oh look, the audio controller that used to be a nice separate device is now also buried into a multifunction device without isolation, no more assigning that to a VM.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc.<br> [AMD/ATI] Hawaii PRO [Radeon R9 290]</blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Device aac8<br></blockquote><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"> 02:00.0 USB controller: ASMedia Technology Inc. Device 1242<br> 03:00.0 Ethernet controller: Qualcomm Atheros Device e0a1 (rev 10)<br> 04:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5721<br> Gigabit Ethernet PCI Express (rev 21)<br> <br> <br> If its not problem of unsafe interrupts, does anyone know why this can<br> happen?<br></blockquote><div><br></div><div>I don't see any reason to implicate unsafe interrupts. It would be nice to understand exactly what happens without specifying unsafe interrupts. It is possible to turn off interrupt remapping support with kernel config options and boot options, so please make sure you have CONFIG_IRQ_REMAP=y in your kernel and aren't disabling it on the commandline.</div></div></div></div>