[vfio-users] problem with passthrough and unsafe interrupts
Janusz
januszmk6 at gmail.com
Sat Sep 12 18:59:16 UTC 2015
W dniu 12.09.2015 o 19:57, Alex Williamson pisze:
> On Sat, Sep 12, 2015 at 11:04 AM, Janusz <januszmk6 at gmail.com
> <mailto:januszmk6 at gmail.com>> wrote:
>
> Hello,
>
> I have a question about allowing unsafe interrupts, what exactly is
> happening when we allow it? why its unsafe?
>
>
> On a standard PC, Message Signaled Interrupts (MSI) are triggered via
> a DMA write by the device to a special address range. In the early
> revisions of VT-d, the IOMMU did not protect this range, which allowed
> devices to effectively spoof interrupts from other devices and
> potentially run attacks against the host using DMA writes to this
> interrupt block. Later versions of VT-d introduced the interrupt
> remapping feature which, among other things, protects this range so
> that a device can only signal the interrupts programmed for it.
>
> Another thing that interrupt remapping does is provide a translation
> between IOAPIC and X2APIC interrupt domains on the system. Pretty
> much all new Intel processors support X2APIC, which necessitates
> interrupt remapping support. So, I'd be really surprised if your new
> skylake system doesn't have interrupt remapping. You should really
> only need to enable the unsafe interrupts option if you try to assign
> the device, it fails, and dmesg tells you that you need it. And only
> then if you trust your guests not to be malicious to the host.
thanks for explanation
>
>
> I am asking because I am not able to passthrough my gpu without
> allowing
> unsafe interrupts, and have problems in running my virtual machine
> as it
> sometimes rebooting its self before its able to run windows. I
> know that
> there is also problem with using iGPU and not uefi bios, but its also
> happening with OVMF and passing uefi rom for my gpu (except that with
> OVMF sometimes I also get very high load). without passing VGA,
> everyting works fine. Is it possible that its because those unsafe
> interrupts?
> My hardware: i7 6700k, MSI z170a M7, Sapphire R9 290
>
>
> Congratulations, you're the first person to show up with a Skylake
> system, welcome to trailblazing a new Intel platform. When you say
> you're not able to assign the GPU without the unsafe interrupts
> option, does that mean when you run the VM QEMU exits and dmesg reports:
>
> "No interrupt remapping support. Use the module param
> "allow_unsafe_interrupts" to enable VFIO IOMMU support on this platform"
>
> That would be pretty epic if Intel dropped interrupt remapping support
> on Skylake. Or perhaps you just mean that you have something closer
> to working with unsafe interrupts. If the hardware supports interrupt
> remapping, then the unsafe option makes absolutely no difference.
> It's only meant to allow an otherwise un-allowed scenario by making
> the user opt-in to the security risk.
actually I didn't have CONFIG_IRQ_REMAP in kernel config set... now,
when I set vfio_iommu_type1.allow_unsafe_interrupts=0, its working
>
> those are my options for non-uefi:
> -enable-kvm -m 10000 -cpu host -smp 8,cores=4,threads=2,sockets=1
> -device
> ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1
> -device vfio-pci,host=01:00.0,multifunction=on,x-vga=on
> -device vfio-pci,host=01:00.1
>
>
> Looks like you're adding an ioh3420 for absolutely no reason. Same
> with the multifunction option.
didn't know about that, I took it from some howto on wiki.debian.org,
disabled it now
>
>
> and for uefi:
>
> -drive
> if=pflash,format=raw,readonly,file=/home/janusz/uefi/OVMF_CODE.fd
> -drive if=pflash,format=raw,file=/home/janusz/uefi/OVMF_VARS.fd
> -enable-kvm -m 10000 -cpu host -smp 8,cores=4,threads=2,sockets=1
> -device
> vfio-pci,host=01:00.0,romfile=/home/janusz/uefi/uefi-vga.bin,multifunction=on
> -device vfio-pci,host=01:00.1
>
>
> Useless multifunction option again. Why does this one specify a ROM
> file? Does the on-card ROM not support UEFI?
Yes, my GPU doesn't support UEFI, but I found UEFI rom in web for this GPU
>
>
> + options with assigning /dev/sdb and two usb devices
>
> I am using ovmf build recently from master, kernel 4.2.0 and
> qemu-2.4.50
>
>
> My iommu groups:
>
> /sys/kernel/iommu_groups/0/devices/0000:00:00.0
>
>
>
> /sys/kernel/iommu_groups/1/devices/0000:00:01.0
> /sys/kernel/iommu_groups/1/devices/0000:01:00.0
> /sys/kernel/iommu_groups/1/devices/0000:01:00.1
>
>
> So this ought to be the group we care about.
>
>
> /sys/kernel/iommu_groups/2/devices/0000:00:02.0
> /sys/kernel/iommu_groups/3/devices/0000:00:08.0
> /sys/kernel/iommu_groups/4/devices/0000:00:14.0
> /sys/kernel/iommu_groups/4/devices/0000:00:14.2
> /sys/kernel/iommu_groups/5/devices/0000:00:15.0
> /sys/kernel/iommu_groups/5/devices/0000:00:15.1
> /sys/kernel/iommu_groups/6/devices/0000:00:16.0
> /sys/kernel/iommu_groups/7/devices/0000:00:17.0
>
>
>
> /sys/kernel/iommu_groups/8/devices/0000:00:1c.0
> /sys/kernel/iommu_groups/8/devices/0000:00:1c.2
> /sys/kernel/iommu_groups/8/devices/0000:00:1c.7
> /sys/kernel/iommu_groups/8/devices/0000:02:00.0
> /sys/kernel/iommu_groups/8/devices/0000:03:00.0
> /sys/kernel/iommu_groups/8/devices/0000:04:00.0
>
>
> Lovely, all the PCH root ports are grouped together, people looking
> for hardware please take note.
>
>
> /sys/kernel/iommu_groups/9/devices/0000:00:1e.0
> /sys/kernel/iommu_groups/10/devices/0000:00:1f.0
> /sys/kernel/iommu_groups/10/devices/0000:00:1f.2
> /sys/kernel/iommu_groups/10/devices/0000:00:1f.3
> /sys/kernel/iommu_groups/10/devices/0000:00:1f.4
>
> and lspci:
>
> 00:00.0 Host bridge: Intel Corporation Sky Lake Host Bridge/DRAM
> Registers (rev 07)
>
>
>
> 00:01.0 PCI bridge: Intel Corporation Sky Lake PCIe Controller (x16)
> (rev 07)
>
>
>
> 00:02.0 VGA compatible controller: Intel Corporation Sky Lake
> Integrated
> Graphics (rev 06)
>
>
>
> 00:08.0 System peripheral: Intel Corporation Sky Lake Gaussian
> Mixture Model
>
>
>
> 00:14.0 USB controller: Intel Corporation Sunrise Point-H USB 3.0 xHCI
> Controller (rev 31)
> 00:14.2 Signal processing controller: Intel Corporation Sunrise
> Point-H
> Thermal subsystem (rev 31)
>
>
> Don't you like how they've put the USB3 controller on a multifunction
> device with some random thermal management device, without isolation
> of course. Wonder what chaos assigning that to a VM would cause.
>
>
> 00:15.0 Signal processing controller: Intel Corporation Sunrise
> Point-H
> LPSS I2C Controller #0 (rev 31)
> 00:15.1 Signal processing controller: Intel Corporation Sunrise
> Point-H
> LPSS I2C Controller #1 (rev 31)
>
>
>
> 00:16.0 Communication controller: Intel Corporation Sunrise
> Point-H CSME
> HECI #1 (rev 31)
>
>
>
> 00:17.0 SATA controller: Intel Corporation Device a102 (rev 31)
>
>
>
> 00:1c.0 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root
> Port #1 (rev f1)
> 00:1c.2 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root
> Port #3 (rev f1)
> 00:1c.7 PCI bridge: Intel Corporation Sunrise Point-H PCI Express Root
> Port #8 (rev f1)
> 00:1e.0 Signal processing controller: Intel Corporation Sunrise
> Point-H
> LPSS UART #0 (rev 31)
>
>
>
> 00:1f.0 ISA bridge: Intel Corporation Sunrise Point-H LPC Controller
> (rev 31)
> 00:1f.2 Memory controller: Intel Corporation Sunrise Point-H PMC
> (rev 31)
> 00:1f.3 Audio device: Intel Corporation Sunrise Point-H HD Audio
> (rev 31)
> 00:1f.4 SMBus: Intel Corporation Sunrise Point-H SMBus (rev 31)
>
>
> Oh look, the audio controller that used to be a nice separate device
> is now also buried into a multifunction device without isolation, no
> more assigning that to a VM.
>
>
> 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc.
> [AMD/ATI] Hawaii PRO [Radeon R9 290]
>
> 01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI]
> Device aac8
>
>
>
> 02:00.0 USB controller: ASMedia Technology Inc. Device 1242
> 03:00.0 Ethernet controller: Qualcomm Atheros Device e0a1 (rev 10)
> 04:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5721
> Gigabit Ethernet PCI Express (rev 21)
>
>
> If its not problem of unsafe interrupts, does anyone know why this can
> happen?
>
>
> I don't see any reason to implicate unsafe interrupts. It would be
> nice to understand exactly what happens without specifying unsafe
> interrupts. It is possible to turn off interrupt remapping support
> with kernel config options and boot options, so please make sure you
> have CONFIG_IRQ_REMAP=y in your kernel and aren't disabling it on the
> commandline.
So, now with unsafe interrupts turned off it works, but any idea why for
the most of time virtaul machine is restarting itself? when I am trying
OVMF, its restarting sometimes when windows is starting to load (both
windows installation process and installed OS, windows 8 and windows
10), sometimes at tainocore logo, and sometimes this huge cpu usage...
Also I noticed that in boot menu of ovmf its detecting only 2GHz CPU and
6GB ram, but on ram testing at start - its detecting proper number.
Without OVMF I have no idea in what step its restarting as monitor gets
signal only when windows loads, and after windows loads there is no
problem with using VM, but i know its restarting as there is problem
with using iGPU with vga passthrough - colors on one of my monitors are
corrupted and when I restore it with changing mode by xrandr, it breaks
again until windows boots. Also I hear some weird noise in sound when VM
is starting and this noise is also indicating that VM is restarting (I
have pulseaudio). The iGPU problem is also the reason why I want to use
OVMF so I don't have to fix colors or patch kernel and disable some of
iGPU functions, but with OVMF restarting problem happens more frequently.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20150912/02a6a8ae/attachment.htm>
More information about the vfio-users
mailing list