[vfio-users] intel_iommu=on and aacraid / Adaptec 3805

David david283 at gmail.com
Thu Jul 14 00:28:31 UTC 2016 

Ok, Rebooted when i got home and ran the Dmesg command again to save
you a full copy.  This time its full of errors....
I have no idea what changed.

But the errors are for a device address that has no hardware.

I have attached the error log.

# lspci -v -s 03:01.0
**Nothing**

# for iommu_group in $(find /sys/kernel/iommu_groups/ -maxdepth 1
-mindepth 1 -type d); do echo "IOMMU group $(basename
"$iommu_group")"; for device in $(ls -1 "$iommu_group"/devices/); do
echo -n $'\t'; lspci -nns "$device"; done; done
IOMMU group 0
    00:00.0 Host bridge [0600]: Intel Corporation Xeon E7 v2/Xeon E5
v2/Core i7 DMI2 [8086:0e00] (rev 04)
IOMMU group 1
    00:01.0 PCI bridge [0604]: Intel Corporation Xeon E7 v2/Xeon E5
v2/Core i7 PCI Express Root Port 1a [8086:0e02] (rev 04)
IOMMU group 2
    00:01.1 PCI bridge [0604]: Intel Corporation Xeon E7 v2/Xeon E5
v2/Core i7 PCI Express Root Port 1b [8086:0e03] (rev 04)
IOMMU group 3
    00:02.0 PCI bridge [0604]: Intel Corporation Xeon E7 v2/Xeon E5
v2/Core i7 PCI Express Root Port 2a [8086:0e04] (rev 04)
IOMMU group 4
    00:03.0 PCI bridge [0604]: Intel Corporation Xeon E7 v2/Xeon E5
v2/Core i7 PCI Express Root Port 3a [8086:0e08] (rev 04)
IOMMU group 5
    00:03.2 PCI bridge [0604]: Intel Corporation Xeon E7 v2/Xeon E5
v2/Core i7 PCI Express Root Port 3c [8086:0e0a] (rev 04)
IOMMU group 6
    00:05.0 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 VTd/Memory Map/Misc [8086:0e28] (rev 04)
    00:05.2 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 IIO RAS [8086:0e2a] (rev 04)
    00:05.4 PIC [0800]: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core
i7 IOAPIC [8086:0e2c] (rev 04)
IOMMU group 7
    00:11.0 PCI bridge [0604]: Intel Corporation C600/X79 series
chipset PCI Express Virtual Root Port [8086:1d3e] (rev 06)
IOMMU group 8
    00:16.0 Communication controller [0780]: Intel Corporation
C600/X79 series chipset MEI Controller #1 [8086:1d3a] (rev 05)
IOMMU group 9
    00:19.0 Ethernet controller [0200]: Intel Corporation 82579V
Gigabit Network Connection [8086:1503] (rev 06)
IOMMU group 10
    00:1a.0 USB controller [0c03]: Intel Corporation C600/X79 series
chipset USB2 Enhanced Host Controller #2 [8086:1d2d] (rev 06)
IOMMU group 11
    00:1b.0 Audio device [0403]: Intel Corporation C600/X79 series
chipset High Definition Audio Controller [8086:1d20] (rev 06)
IOMMU group 12
    00:1c.0 PCI bridge [0604]: Intel Corporation C600/X79 series
chipset PCI Express Root Port 1 [8086:1d10] (rev b6)
IOMMU group 13
    00:1c.1 PCI bridge [0604]: Intel Corporation C600/X79 series
chipset PCI Express Root Port 2 [8086:1d12] (rev b6)
IOMMU group 14
    00:1c.2 PCI bridge [0604]: Intel Corporation C600/X79 series
chipset PCI Express Root Port 3 [8086:1d14] (rev b6)
IOMMU group 15
    00:1c.3 PCI bridge [0604]: Intel Corporation C600/X79 series
chipset PCI Express Root Port 4 [8086:1d16] (rev b6)
IOMMU group 16
    00:1c.4 PCI bridge [0604]: Intel Corporation C600/X79 series
chipset PCI Express Root Port 5 [8086:1d18] (rev b6)
IOMMU group 17
    00:1c.5 PCI bridge [0604]: Intel Corporation C600/X79 series
chipset PCI Express Root Port 6 [8086:1d1a] (rev b6)
IOMMU group 18
    00:1c.7 PCI bridge [0604]: Intel Corporation C600/X79 series
chipset PCI Express Root Port 8 [8086:1d1e] (rev b6)
IOMMU group 19
    00:1d.0 USB controller [0c03]: Intel Corporation C600/X79 series
chipset USB2 Enhanced Host Controller #1 [8086:1d26] (rev 06)
IOMMU group 20
    00:1e.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge
[8086:244e] (rev a6)
IOMMU group 21
    00:1f.0 ISA bridge [0601]: Intel Corporation C600/X79 series
chipset LPC Controller [8086:1d41] (rev 06)
    00:1f.2 RAID bus controller [0104]: Intel Corporation SATA
Controller [RAID mode] [8086:2822] (rev 06)
    00:1f.3 SMBus [0c05]: Intel Corporation C600/X79 series chipset
SMBus Host Controller [8086:1d22] (rev 06)
IOMMU group 22
    07:00.0 SATA controller [0106]: Marvell Technology Group Ltd.
88SE9230 PCIe SATA 6Gb/s Controller [1b4b:9230] (rev 10)
IOMMU group 23
    01:00.0 VGA compatible controller [0300]: Advanced Micro Devices,
Inc. [AMD/ATI] Caicos [Radeon HD 6450/7450/8450 / R5 230 OEM]
[1002:6779]
    01:00.1 Audio device [0403]: Advanced Micro Devices, Inc.
[AMD/ATI] Caicos HDMI Audio [Radeon HD 6400 Series] [1002:aa98]
IOMMU group 24
    02:00.0 PCI bridge [0604]: Intel Corporation 80333 Segment-A PCIe
Express to PCI-X bridge [8086:0370]
    02:00.2 PCI bridge [0604]: Intel Corporation 80333 Segment-B PCIe
Express to PCI-X bridge [8086:0372]
    03:0e.0 RAID bus controller [0104]: Adaptec AAC-RAID [9005:0285]
IOMMU group 25
    05:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK104
[GeForce GTX 760] [10de:1187] (rev a1)
    05:00.1 Audio device [0403]: NVIDIA Corporation GK104 HDMI Audio
Controller [10de:0e0a] (rev a1)
IOMMU group 26
    09:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042
SuperSpeed USB Host Controller [1b21:1042]
IOMMU group 27
    0a:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042
SuperSpeed USB Host Controller [1b21:1042]
IOMMU group 28
    0b:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042
SuperSpeed USB Host Controller [1b21:1042]
IOMMU group 29
    0c:00.0 Network controller [0280]: Broadcom Corporation BCM4352
802.11ac Wireless Network Adapter [14e4:43b1] (rev 03)
IOMMU group 30
    0d:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062
Serial ATA Controller [1b21:0612] (rev 01)
IOMMU group 31
    0e:00.0 Ethernet controller [0200]: Realtek Semiconductor Co.,
Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
[10ec:8168] (rev 11)
IOMMU group 32
    0f:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062
Serial ATA Controller [1b21:0612] (rev 01)
IOMMU group 33
    ff:08.0 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 QPI Link 0 [8086:0e80] (rev 04)
IOMMU group 34
    ff:09.0 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 QPI Link 1 [8086:0e90] (rev 04)
IOMMU group 35
    ff:0a.0 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Power Control Unit 0 [8086:0ec0] (rev 04)
    ff:0a.1 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Power Control Unit 1 [8086:0ec1] (rev 04)
    ff:0a.2 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Power Control Unit 2 [8086:0ec2] (rev 04)
    ff:0a.3 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Power Control Unit 3 [8086:0ec3] (rev 04)
IOMMU group 36
    ff:0b.0 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 UBOX Registers [8086:0e1e] (rev 04)
    ff:0b.3 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 UBOX Registers [8086:0e1f] (rev 04)
IOMMU group 37
    ff:0c.0 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Unicast Registers [8086:0ee0] (rev 04)
    ff:0c.1 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Unicast Registers [8086:0ee2] (rev 04)
    ff:0c.2 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Unicast Registers [8086:0ee4] (rev 04)
IOMMU group 38
    ff:0d.0 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Unicast Registers [8086:0ee1] (rev 04)
    ff:0d.1 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Unicast Registers [8086:0ee3] (rev 04)
    ff:0d.2 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Unicast Registers [8086:0ee5] (rev 04)
IOMMU group 39
    ff:0e.0 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Home Agent 0 [8086:0ea0] (rev 04)
    ff:0e.1 Performance counters [1101]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Home Agent 0 [8086:0e30] (rev 04)
IOMMU group 40
    ff:0f.0 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Target
Address/Thermal Registers [8086:0ea8] (rev 04)
    ff:0f.1 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 RAS Registers
[8086:0e71] (rev 04)
    ff:0f.2 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Channel Target
Address Decoder Registers [808... (rev 04)
    ff:0f.3 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Channel Target
Address Decoder Registers [808... (rev 04)
    ff:0f.4 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Channel Target
Address Decoder Registers [808... (rev 04)
    ff:0f.5 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Integrated Memory Controller 0 Channel Target
Address Decoder Registers [808... (rev 04)
IOMMU group 41
    ff:10.0 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3
Thermal Control 0 [8086:0eb0] (rev 04)
    ff:10.1 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3
Thermal Control 1 [8086:0eb1] (rev 04)
    ff:10.2 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 ERROR
Registers 0 [8086:0eb2] (rev 04)
    ff:10.3 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 ERROR
Registers 1 [8086:0eb3] (rev 04)
    ff:10.4 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3
Thermal Control 2 [8086:0eb4] (rev 04)
    ff:10.5 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3
Thermal Control 3 [8086:0eb5] (rev 04)
    ff:10.6 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 ERROR
Registers 2 [8086:0eb6] (rev 04)
    ff:10.7 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Integrated Memory Controller 1 Channel 0-3 ERROR
Registers 3 [8086:0eb7] (rev 04)
IOMMU group 42
    ff:13.0 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 R2PCIe [8086:0e1d] (rev 04)
    ff:13.1 Performance counters [1101]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 R2PCIe [8086:0e34] (rev 04)
    ff:13.4 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 QPI Ring Registers [8086:0e81] (rev 04)
    ff:13.5 Performance counters [1101]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 QPI Ring Performance Ring Monitoring [8086:0e36]
(rev 04)
IOMMU group 43
    ff:16.0 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 System Address Decoder [8086:0ec8] (rev 04)
    ff:16.1 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Broadcast Registers [8086:0ec9] (rev 04)
    ff:16.2 System peripheral [0880]: Intel Corporation Xeon E7
v2/Xeon E5 v2/Core i7 Broadcast Registers [8086:0eca] (rev 04)

On Wed, Jul 13, 2016 at 11:03 AM, Alex Williamson
<alex.williamson at redhat.com> wrote:
> On Tue, 12 Jul 2016 20:29:13 -0500
> David <david283 at gmail.com> wrote:
>
>> the command "dmesg | grep -i dma"  Give no results.  Is there anything
>> else i should check Alex?
>>
>> If the solution to this is to submit a bug report, who should i send
>> it too?  I believe the AACRAID driver is built into the Linux kernel.
>
> How about sending the full, uncut, unfiltered dmesg out first?  Have
> you tried adding iommu=pt as a boot option?  Thanks,
>
> Alex
>
>> On Tue, Jul 12, 2016 at 9:51 AM, Alex Williamson
>> <alex.williamson at redhat.com> wrote:
>> > On Sun, 10 Jul 2016 13:17:41 -0500
>> > David <david283 at gmail.com> wrote:
>> >
>> >> I have run into a strange problem, while setting up my Fedora 24 box for
>> >> KVM, i noticed that my raid array stopped showing as an available drive.
>> >> After a lot of troubleshooting and reinstalling linux to this PC 2 more
>> >> times, i have narrowed it down to one setting in my GRUB config.
>> >>  intel_iommu=on
>> >>
>> >> Just taking that one setting out and rebuilding my grub2-efi.cfg will make
>> >> the array readable again.  When IOMMU is on, the system can see that there
>> >> is a raid card and array, but it lists the partition table as unknown.  It
>> >> also will not successfully create a new GPT partition table on the array.
>> >> With IOMMU off, it can read the partition table and partition fine, read
>> >> and write data, and everything works fine.
>> >>
>> >> My raid is 4 1TB disks in Raid 10e, GPT partition formatted NTFS.
>> >>
>> >> $ lspci -v -s 03:0e.0
>> >> 03:0e.0 RAID bus controller: Adaptec AAC-RAID
>> >>     Subsystem: Adaptec 3805
>> >>     Flags: bus master, stepping, 66MHz, medium devsel, latency 32, IRQ 57
>> >>     Memory at fa600000 (64-bit, non-prefetchable) [size=2M]
>> >>     Expansion ROM at fa800000 [disabled] [size=256K]
>> >>     Capabilities: <access denied>
>> >>     Kernel driver in use: aacraid
>> >>     Kernel modules: aacraid
>> >
>> >
>> > No DMAR faults in dmesg?  There's a fair chance this is one of those
>> > devices that does DMA with the wrong requester ID and therefore
>> > enabling the IOMMU prevents it from working.  We have a mechanism in
>> > the kernel to handle such DMA aliasing if you can verify the issue.
>> > Thanks,
>> >
>> > Alex
>>
>>
>>
>



-- 
David
david283 at gmail.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: errorlog2
Type: application/octet-stream
Size: 280557 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160713/77ab04d6/attachment.obj>

More information about the vfio-users mailing list