[vfio-users] RocketRaid 1144c / AsMedia 1042a

[Alex Williamson]

On Fri, Jun 17, 2016 at 9:56 PM, Nick Rodgers <dotbatman at nerdfortress.com>
wrote:

> Almost a year and two more 4-controller usb 3.0 cards later, I still
> haven't found one that does what I'm after. I purchased the StarTech
> PEXUSB3S44V and the Sonnet Allegro Pro. Both devices get their own IOMMU
> group for each controller without the ACS override patch. The StarTech
> device seems to pass through okay, but I get in windows I get a code 10 on
> its root hub.
>

It seems like I've seen this with NEC USB3 devices, have you tried using
pci-stub to prevent attaching the host xhci drivers to the device?  I have
a typical single controller NEC card that works more often than not with
that change, but I'm not sure if it's one of the cheap clone chips or a
real NEC part.  It has less than desirable behavior on bare metal too.


> The Sonnet Allegro Pro passes through perfectly, until I restart or
> shutdown the VM. After the passed through controller's entry in lspci -vvvv
> changes from
>
> 04:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev
> 10) (prog-if 30 [XHCI])
>         Subsystem: Sonnet Technologies, Inc. Device 6e31
>         Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> ParErr- Stepping- SERR- FastB2B- DisINTx+
>         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
> <TAbort- <MAbort- >SERR- <PERR- INTx-
>         Latency: 0, Cache Line Size: 64 bytes
>         Interrupt: pin A routed to IRQ 26
>         NUMA node: 0
>         Region 0: Memory at fe500000 (64-bit, non-prefetchable) [size=64K]
>         Region 2: Memory at fe511000 (64-bit, non-prefetchable) [size=4K]
>         Region 4: Memory at fe510000 (64-bit, non-prefetchable) [size=4K]
>         Capabilities: [40] Power Management version 3
>                 Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA
> PME(D0+,D1+,D2-,D3hot+,D3cold-)
>                 Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
>

NoSoftRst- == PM reset is available.


>         Capabilities: [50] MSI: Enable- Count=1/8 Maskable- 64bit+
>                 Address: 0000000000000000  Data: 0000
>         Capabilities: [70] Express (v2) Endpoint, MSI 00
>                 DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s
> <64ns, L1 <1us
>                         ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
> SlotPowerLimit 25.000W
>

Does not support FLR.


>                 DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
> Unsupported-
>                         RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
>                         MaxPayload 128 bytes, MaxReadReq 512 bytes
>                 DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr-
> TransPend-
>                 LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit
> Latency L0s unlimited, L1 unlimited
>                         ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp-
>                 LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-
>                         ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
>                 LnkSta: Speed 5GT/s, Width x1, TrErr- Train- SlotClk+
> DLActive- BWMgmt- ABWMgmt-
>                 DevCap2: Completion Timeout: Not Supported, TimeoutDis+,
> LTR-, OBFF Not Supported
>                 DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-,
> LTR-, OBFF Disabled
>                 LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance-
> SpeedDis-
>                          Transmit Margin: Normal Operating Range,
> EnterModifiedCompliance- ComplianceSOS-
>                          Compliance De-emphasis: -6dB
>                 LnkSta2: Current De-emphasis Level: -6dB,
> EqualizationComplete-, EqualizationPhase1-
>                          EqualizationPhase2-, EqualizationPhase3-,
> LinkEqualizationRequest-
>         Capabilities: [b0] MSI-X: Enable+ Count=8 Masked-
>                 Vector table: BAR=2 offset=00000000
>                 PBA: BAR=4 offset=00000000
>         Capabilities: [100 v1] Advanced Error Reporting
>                 UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt-
> RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
>                 UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt-
> RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
>                 UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt-
> RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
>                 CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout-
> NonFatalErr-
>                 CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout-
> NonFatalErr+
>                 AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+
> ChkEn-
>         Kernel driver in use: vfio-pci
>         Kernel modules: xhci_pci
>
> to
>
> 04:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev
> ff) (prog-if ff)
>         !!! Unknown header type 7f
>         Kernel driver in use: vfio-pci
>         Kernel modules: xhci_pci
>

IIRC we'll generally opt for a bus reset rather than using PM reset, but
this is indicative of a device that isn't returning from a PCI bus reset,
config space is returning -1 when read.  You can try quirking the device to
avoid bus resets, add a line similar to this for your device:

http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/pci/quirks.c#n3198
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160620/7ebd1caa/attachment.htm>