[vfio-users] Passthrough of SAS2008-based card on E3-1225v3 fails due to DMA mapping error
Johannes Falke
johannesfalke at googlemail.com
Tue Nov 7 18:44:12 UTC 2017
Hey everyone,
I use Proxmox to manage my VMs, and one of the VMs requires a
passed-through HBA. Up until today I've been using Proxmox 3.4 with kernel
2.6.32, which did not use VFIO for passthrough. This set-up worked fine
using an IBM M1015 (SAS2008) flashed to 9211-8i IT mode, so the card isn't
faulty and is possible to be passed through. Today I updated to the latest
version of Proxmox where passthrough is always done via VFIO as far as I
can tell.
However, with VFIO it is not possible to pass through this card. A dummy
command for pass-through returns:
# qemu-system-x86_64 -enable-kvm -m 10G -net none -monitor stdio -serial
> none -parallel none -vnc :1 -device vfio-pci,host=1:00.0 -S
> QEMU 2.9.1 monitor - type 'help' for more information
> (qemu) c
> (qemu) qemu-system-x86_64: VFIO_MAP_DMA: -14
> qemu-system-x86_64: vfio_dma_map(0x7f548f0a1fc0, 0xfebd0000, 0x2000,
> 0x7f54a909d000) = -14 (Bad address)
> qemu: hardware error: vfio: DMA mapping failed, unable to continue
> CPU #0:
> EAX=00000103 EBX=80001804 ECX=00000103 EDX=00000cfc
> ESI=00001800 EDI=00000000 EBP=bffab6b0 ESP=00006d34
> EIP=000eda90 EFL=00000046 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
> ES =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA]
> CS =0008 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA]
> SS =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA]
> DS =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA]
> FS =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA]
> GS =0010 00000000 ffffffff 00c09300 DPL=0 DS [-WA]
> LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
> TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy
> GDT= 000f7070 00000037
> IDT= 000f70ae 00000000
> CR0=00000011 CR2=00000000 CR3=00000000 CR4=00000000
> DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
> DR3=0000000000000000
> DR6=00000000ffff0ff0 DR7=0000000000000400
> EFER=0000000000000000
> FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
> FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
> FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
> FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
> FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
> XMM00=00000000000000000000000000000000
> XMM01=00000000000000000000000000000000
> XMM02=00000000000000000000000000000000
> XMM03=00000000000000000000000000000000
> XMM04=00000000000000000000000000000000
> XMM05=00000000000000000000000000000000
> XMM06=00000000000000000000000000000000
> XMM07=00000000000000000000000000000000
> Aborted
>
This has been previously reported on this mailing list (
https://www.redhat.com/archives/vfio-users/2016-May/msg00054.html), but no
solution was found for the poster.
It seems that on Fedora this passthrough works fine (
https://www.redhat.com/archives/vfio-users/2016-May/msg00246.html), but in
Ubuntu and Proxmox (also Debian-based), it doesn't. Other cards, such as
the 9207-8i (SAS2308 chip) also seem to work (
https://forum.proxmox.com/threads/promxox4-4-qm-status-stopped-for-passthrough-of-ibm1015.32841/#post-169018
).
Any ideas on how to get this card working? I'd be grateful for any input,
and I'm not alone in this combination of hardware.
For completeness:
root at pve:~# uname -a
Linux pve 4.13.4-1-pve #1 SMP PVE 4.13.4-26 (Mon, 6 Nov 2017 11:23:55
+0100) x86_64 GNU/Linux
and
01:00.0 Serial Attached SCSI controller [0107]: LSI Logic / Symbios Logic
SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] [1000:0072] (rev 03)
Subsystem: LSI Logic / Symbios Logic SAS2008 PCI-Express Fusion-MPT
SAS-2 [Falcon] [1000:3020]
Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 16
Region 0: I/O ports at e000 [disabled] [size=256]
Region 1: Memory at df6c0000 (64-bit, non-prefetchable) [disabled]
[size=16K]
Region 3: Memory at df280000 (64-bit, non-prefetchable) [disabled]
[size=256K]
Expansion ROM at df200000 [disabled] [size=512K]
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D3 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [68] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 4096 bytes, PhantFunc 0, Latency L0s
<64ns, L1 <1us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
SlotPowerLimit 0.000W
DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
Unsupported-
RlxdOrd- ExtTag+ PhantFunc- AuxPwr- NoSnoop+
FLReset-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr-
TransPend-
LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s, Exit
Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 5GT/s, Width x8, TrErr- Train- SlotClk+
DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range BC, TimeoutDis+, LTR-,
OBFF Not Supported
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-,
LTR-, OBFF Disabled
LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance-
SpeedDis-
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -6dB,
EqualizationComplete-, EqualizationPhase1-
EqualizationPhase2-, EqualizationPhase3-,
LinkEqualizationRequest-
Capabilities: [d0] Vital Product Data
pcilib: sysfs_read_vpd: read failed: Input/output error
Not readable
Capabilities: [a8] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [c0] MSI-X: Enable- Count=15 Masked-
Vector table: BAR=1 offset=00002000
PBA: BAR=1 offset=00003800
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt-
RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt-
RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt-
RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr+
AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+
ChkEn-
Capabilities: [138 v1] Power Budgeting <?>
Capabilities: [150 v1] Single Root I/O Virtualization (SR-IOV)
IOVCap: Migration-, Interrupt Message Number: 000
IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy-
IOVSta: Migration-
Initial VFs: 16, Total VFs: 16, Number of VFs: 0, Function
Dependency Link: 00
VF offset: 1, stride: 1, Device ID: 0072
Supported Page Size: 00000553, System Page Size: 00000001
Region 0: Memory at 00000000df6c4000 (64-bit,
non-prefetchable)
Region 2: Memory at 00000000df2c0000 (64-bit,
non-prefetchable)
VF Migration: offset: 00000000, BIR: 0
Capabilities: [190 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 0
ARICtl: MFVC- ACS-, Function Group: 0
Kernel driver in use: vfio-pci
Kernel modules: mpt3sas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20171107/bb46cd65/attachment.htm>
More information about the vfio-users
mailing list