[vfio-users] VFIO not claiming device

[Brett Peckinpaugh]

To start I am now running Arch with the VFIO kernel, I did not compile 
with the i915, my machine failed to boot with that last time.  Currently 
I can't seem to get VFIO to claim the devices I want to pass-through.  
You can see kernel driver is still nvidia.  I added my configs below.  I 
even tried a script.  Every time I modify mkinitcpio.conf I regenerate 
my intramfd with mkinitcpio -p linux-vfio

What am I missing?

lspci -nnk -d 10de:1087

    02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF110
    [GeForce GTX 560 Ti 448 Cores] [10de:1087] (rev a1)
             Subsystem: NVIDIA Corporation Device [10de:0000]
             Kernel driver in use: nvidia
             Kernel modules: nouveau, nvidia_drm, nvidia



/etc/mkinitcpio.conf

    MODULES="vfio vfio_iommu_type1 vfio_pci vfio_virqfd"

kernel boot options.

    intel_iommu=on pcie_acs_override=downstream rd.driver.pre=vfio-pci

/etc/modprobe.d/VFIO.conf

    options vfio-pci ids=10de:1087,10de:0e09

modinfo vfio-pci

    filename:
    /lib/modules/4.5.1-1-vfio/kernel/drivers/vfio/pci/vfio-pci.ko.gz
    description:    VFIO PCI - User Level meta-driver
    author:         Alex Williamson <alex.williamson at redhat.com>
    license:        GPL v2
    version:        0.2
    srcversion:     C2A1332D492B2BE5378D1E1
    depends:        vfio,irqbypass,vfio_virqfd
    intree:         Y
    vermagic:       4.5.1-1-vfio SMP preempt mod_unload modversions
    parm:           ids:Initial PCI IDs to add to the vfio driver,
    format is
    "vendor:device[:subvendor[:subdevice[:class[:class_mask]]]]" and
    multiple comma separated entries can be specified (string)
    parm:           nointxmask:Disable support for PCI 2.3 style INTx
    masking.  If this resolves problems for specific devices, report
    lspci -vvvxxx to linux-pci at vger.kernel.org so the device can be
    fixed automatically via the broken_intx_masking flag. (bool)
    parm:           disable_vga:Disable VGA resource access through
    vfio-pci (bool)
    parm:           disable_idle_d3:Disable using the PCI D3 low power
    state for idle, unused devices (bool)

dmesg|grep -e DMAR -e IOMMU

    [    0.000000] Warning: PCIe ACS overrides enabled; This may allow
    non-IOMMU protected peer-to-peer DMA
    [    0.000000] ACPI: DMAR 0x0000000087F4D9F8 000070 (v01 INTEL
    SKL      00000001 INTL 00000001)
    [    0.000000] DMAR: IOMMU enabled
    [    0.033182] DMAR: Host address width 39
    [    0.033183] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
    [    0.033187] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap
    d2008c40660462 ecap f050da
    [    0.033188] DMAR: RMRR base: 0x00000087906000 end: 0x00000087925fff
    [    0.033189] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed90000 IOMMU 0
    [    0.033190] DMAR-IR: HPET id 0 under DRHD base 0xfed90000
    [    0.033191] DMAR-IR: x2apic is disabled because BIOS sets x2apic
    opt out bit.
    [    0.033191] DMAR-IR: Use 'intremap=no_x2apic_optout' to override
    the BIOS setting.
    [    0.034540] DMAR-IR: Enabled IRQ remapping in xapic mode
    [    0.352192] DMAR: No ATSR found
    [    0.352245] DMAR: dmar0: Using Queued invalidation
    [    0.352251] DMAR: Setting RMRR:
    [    0.352258] DMAR: Setting identity map for device 0000:00:14.0
    [0x87906000 - 0x87925fff]
    [    0.352262] DMAR: Prepare 0-16MiB unity mapping for LPC
    [    0.352266] DMAR: Setting identity map for device 0000:00:1f.0
    [0x0 - 0xffffff]
    [    0.352270] DMAR: Intel(R) Virtualization Technology for Directed I/O


Incase this helps, here are my IOMMU groupings

    IOMMU group 0
             00:00.0 Host bridge [0600]: Intel Corporation Skylake Host
    Bridge/DRAM Registers [8086:191f] (rev 07)
    IOMMU group 1
             00:01.0 PCI bridge [0604]: Intel Corporation Skylake PCIe
    Controller (x16) [8086:1901] (rev 07)
    IOMMU group 2
             00:01.1 PCI bridge [0604]: Intel Corporation Skylake PCIe
    Controller (x8) [8086:1905] (rev 07)
    IOMMU group 3
             00:14.0 USB controller [0c03]: Intel Corporation Sunrise
    Point-H USB 3.0 xHCI Controller [8086:a12f] (rev 31)
             00:14.2 Signal processing controller [1180]: Intel
    Corporation Sunrise Point-H Thermal subsystem [8086:a131] (rev 31)
    IOMMU group 4
             00:16.0 Communication controller [0780]: Intel Corporation
    Sunrise Point-H CSME HECI #1 [8086:a13a] (rev 31)
    IOMMU group 5
             00:17.0 SATA controller [0106]: Intel Corporation Sunrise
    Point-H SATA controller [AHCI mode] [8086:a102] (rev 31)
    IOMMU group 6
             00:1b.0 PCI bridge [0604]: Intel Corporation Sunrise
    Point-H PCI Root Port #17 [8086:a167] (rev f1)
    IOMMU group 7
             00:1c.0 PCI bridge [0604]: Intel Corporation Sunrise
    Point-H PCI Express Root Port #1 [8086:a110] (rev f1)
    IOMMU group 8
             00:1c.4 PCI bridge [0604]: Intel Corporation Sunrise
    Point-H PCI Express Root Port #5 [8086:a114] (rev f1)
    IOMMU group 9
             00:1c.5 PCI bridge [0604]: Intel Corporation Sunrise
    Point-H PCI Express Root Port #6 [8086:a115] (rev f1)
    IOMMU group 10
             00:1d.0 PCI bridge [0604]: Intel Corporation Sunrise
    Point-H PCI Express Root Port #9 [8086:a118] (rev f1)
    IOMMU group 11
             00:1d.4 PCI bridge [0604]: Intel Corporation Sunrise
    Point-H PCI Express Root Port #13 [8086:a11c] (rev f1)
    IOMMU group 12
             00:1f.0 ISA bridge [0601]: Intel Corporation Sunrise
    Point-H LPC Controller [8086:a145] (rev 31)
             00:1f.2 Memory controller [0580]: Intel Corporation Sunrise
    Point-H PMC [8086:a121] (rev 31)
             00:1f.3 Audio device [0403]: Intel Corporation Sunrise
    Point-H HD Audio [8086:a170] (rev 31)
             00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-H
    SMBus [8086:a123] (rev 31)
    IOMMU group 13
             00:1f.6 Ethernet controller [0200]: Intel Corporation
    Ethernet Connection (2) I219-V [8086:15b8] (rev 31)
    IOMMU group 14
             01:00.0 VGA compatible controller [0300]: NVIDIA
    Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)
             01:00.1 Audio device [0403]: NVIDIA Corporation GM204 High
    Definition Audio Controller [10de:0fbb] (rev a1)
    IOMMU group 15
             02:00.0 VGA compatible controller [0300]: NVIDIA
    Corporation GF110 [GeForce GTX 560 Ti 448 Cores] [10de:1087] (rev a1)
             02:00.1 Audio device [0403]: NVIDIA Corporation GF110 High
    Definition Audio Controller [10de:0e09] (rev a1)
    IOMMU group 16
             04:00.0 PCI bridge [0604]: Intel Corporation DSL6540
    Thunderbolt 3 Bridge [Alpine Ridge 4C 2015] [8086:1578]
    IOMMU group 17
             05:00.0 PCI bridge [0604]: Intel Corporation DSL6540
    Thunderbolt 3 Bridge [Alpine Ridge 4C 2015] [8086:1578]
    IOMMU group 18
             05:01.0 PCI bridge [0604]: Intel Corporation DSL6540
    Thunderbolt 3 Bridge [Alpine Ridge 4C 2015] [8086:1578]
    IOMMU group 19
             05:02.0 PCI bridge [0604]: Intel Corporation DSL6540
    Thunderbolt 3 Bridge [Alpine Ridge 4C 2015] [8086:1578]
    IOMMU group 20
             05:04.0 PCI bridge [0604]: Intel Corporation DSL6540
    Thunderbolt 3 Bridge [Alpine Ridge 4C 2015] [8086:1578]
    IOMMU group 21
             08:00.0 USB controller [0c03]: Intel Corporation DSL6540
    USB 3.1 Controller [Alpine Ridge] [8086:15b6]
    IOMMU group 22
             0a:00.0 Ethernet controller [0200]: Intel Corporation I211
    Gigabit Network Connection [8086:1539] (rev 03)
    IOMMU group 23
             0b:00.0 SATA controller [0106]: ASMedia Technology Inc.
    ASM1062 Serial ATA Controller [1b21:0612] (rev 02)





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160422/d06c708d/attachment.htm>