[vfio-users] vfio pass through only works once after reboot - more tests

[Karsten Elfenbein]

Hi,

I just checked the kernel config and after enabling PCI_MMCONFIG the
capabilities appeared and the error messages disappeared.
The problem with the card only working once per reboot is still there :(


Thanks
Karsten

2015-09-17 23:52 GMT+02:00 Alex Williamson <alex.williamson at redhat.com>:
> On Thu, 2015-09-17 at 23:34 +0200, Karsten Elfenbein wrote:
>> 06:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce
>> GTX 970] (rev a1) (prog-if 00 [VGA controller])
>>         Subsystem: Device 196e:1131
>>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
>> ParErr- Stepping- SERR- FastB2B- DisINTx-
>>         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
>> <TAbort- <MAbort- >SERR- <PERR- INTx-
>>         Latency: 0
>>         Interrupt: pin A routed to IRQ 30
>>         Region 0: Memory at f9000000 (32-bit, non-prefetchable) [size=16M]
>>         Region 1: Memory at d0000000 (64-bit, prefetchable) [size=256M]
>>         Region 3: Memory at e0000000 (64-bit, prefetchable) [size=32M]
>>         Region 5: I/O ports at d000 [size=128]
>>         [virtual] Expansion ROM at fa000000 [disabled] [size=512K]
>>         Capabilities: [60] Power Management version 3
>>                 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
>> PME(D0-,D1-,D2-,D3hot-,D3cold-)
>>                 Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
>>         Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
>>                 Address: 00000000fee00398  Data: 0000
>>         Capabilities: [78] Express (v2) Legacy Endpoint, MSI 00
>>                 DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s
>> unlimited, L1 <64us
>>                         ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
>>                 DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
>> Unsupported-
>>                         RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop-
>>                         MaxPayload 256 bytes, MaxReadReq 512 bytes
>>                 DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq-
>> AuxPwr- TransPend-
>>                 LnkCap: Port #0, Speed 5GT/s, Width x16, ASPM L0s L1,
>> Exit Latency L0s <512ns, L1 <4us
>>                         ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
>>                 LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- CommClk+
>>                         ExtSynch- ClockPM+ AutWidDis- BWInt- AutBWInt-
>>                 LnkSta: Speed 5GT/s, Width x16, TrErr- Train- SlotClk+
>> DLActive- BWMgmt- ABWMgmt-
>>                 DevCap2: Completion Timeout: Range AB, TimeoutDis+,
>> LTR+, OBFF Via message
>>                 DevCtl2: Completion Timeout: 50us to 50ms,
>> TimeoutDis-, LTR-, OBFF Disabled
>>                 LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
>>                          Transmit Margin: Normal Operating Range,
>> EnterModifiedCompliance- ComplianceSOS-
>>                          Compliance De-emphasis: -6dB
>>                 LnkSta2: Current De-emphasis Level: -6dB,
>> EqualizationComplete-, EqualizationPhase1-
>>                          EqualizationPhase2-, EqualizationPhase3-,
>> LinkEqualizationRequest-
>
>
> Something very strange here, where are your extended PCIe capabilities?
> My GTX750 lists:
>
>         Capabilities: [100] Virtual Channel
>         Capabilities: [250] Latency Tolerance Reporting
>         Capabilities: [258] L1 PM Substates
>         Capabilities: [128] Power Budgeting <?>
>         Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
>         Capabilities: [900] #19
>
> Nothing in your system has extended capabilities.  That sort of explains
> the -EFAULT, though maybe it implies a bug elsewhere.  Can you pastebin
> your full dmesg?
>
>>         Kernel driver in use: nvidia
>>         Kernel modules: nvidia
>
> Hmm, keep in mind to try w/o nvidia on the host if we don't make
> progress.
>