<div dir="ltr"><div>first: I use proxmox 5 (pvetest)</div><div> </div>good evening, I'm succesfully passing through a nvidia board to my q35 guest but I'm getting host-freezes after starting the vm with a hostpci1: 04:00.0 <div> </div><div>03:00.0 PCI bridge: Texas Instruments XIO2000(A)/XIO2200A PCI Express-to-PCI Bridge (rev 03) 04:00.0 FireWire (IEEE 1394): Texas Instruments XIO2200A IEEE-1394a-2000 Controller (PHY/Link) (rev 01)</div><div> </div><div>I've already set the corresponding pci-stub arguments to the kernel and checked all that is ok. </div><div> </div><div>I managed to overcome errors on 03:00 not biding when trying to pass the 04:00 device but as soon as the vm starts, the host gets completely frozen. I have to stop my lvm group (running over a raid0) when I do tests because the raid breaks when the host freezes, there is no log/error produced when the crash happens.</div><div> </div><div>I've read that a IOMMU group shared by a device and the pcie bridge produces vfio to try bind the bridge to the vm and that is not possible, but there is also a PCH device produced as a workaround by the kernel, which I don't know how to pass to the vm, as tried in the pve-q35.cfg</div><div> </div><div><div>lspci -tv </div><div>+-1c.4-[03-04]----00.0-[04]----00.0 Texas Instruments XIO2200A IEEE-1394a-2000 Controller (PHY/Link)<div> </div><div>lspci -tnv</div><div>+-1c.4-[03-04]----00.0-[04]----00.0 104c:8235</div><div> </div><div>lspci -vvvv -s 04:00</div><div>04:00.0 FireWire (IEEE 1394): Texas Instruments XIO2200A IEEE-1394a-2000 Controller (PHY/Link) (rev 01) (prog-if 10 [OHCI]) Subsystem: Texas Instruments XIO2200A IEEE-1394a-2000 Controller (PHY/Link) Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Interrupt: pin A routed to IRQ 11 Region 0: Memory at f7904000 (32-bit, non-prefetchable) [disabled] [size=2K] Region 1: Memory at f7900000 (32-bit, non-prefetchable) [disabled] [size=16K] Capabilities: [44] Power Management version 2 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Kernel driver in use: pci-stub Kernel modules: firewire_ohci</div><div> </div><div>lspci -vvvv -s 03:00</div><div>03:00.0 PCI bridge: Texas Instruments XIO2000(A)/XIO2200A PCI Express-to-PCI Bridge (rev 03) (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Bus: primary=03, secondary=04, subordinate=04, sec-latency=32 I/O behind bridge: 0000f000-00000fff Memory behind bridge: f7900000-f79fffff Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort+ <SERR- <PERR- BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [50] Power Management version 2 Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Bridge: PM- B3+ Capabilities: [60] MSI: Enable- Count=1/16 Maskable- 64bit+ Address: 0000000000000000 Data: 0000 Capabilities: [80] Subsystem: Device 0000:0000 Capabilities: [90] Express (v1) PCI-Express to PCI/PCI-X Bridge, MSI 00 DevCap: MaxPayload 512 bytes, PhantFunc 0 ExtTag- AttnBtn- AttnInd- PwrInd- RBE- SlotPowerLimit 10.000W DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+ BrConfRtry- MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq- AuxPwr+ TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- Kernel modules: shpchp<div> </div><div>lspci -vvvv -s 1c.4</div><div>00:1c.4 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #5 (rev d5) (prog-if 00 [Normal decode]) Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0, Cache Line Size: 64 bytes Interrupt: pin A routed to IRQ 28 Bus: primary=00, secondary=03, subordinate=04, sec-latency=0 I/O behind bridge: 0000f000-00000fff Memory behind bridge: f7900000-f79fffff Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR- BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [40] Express (v2) Root Port (Slot+), MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0 ExtTag- RBE+ DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend- LnkCap: Port #5, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <512ns, L1 <16us ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp- LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+ DLActive+ BWMgmt+ ABWMgmt- SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- Slot #4, PowerLimit 10.000W; Interlock- NoCompl+ SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock- Changed: MRL- PresDet- LinkState- RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- DevCap2: Completion Timeout: Range ABC, TimeoutDis+, LTR+, OBFF Not Supported ARIFwd- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR+, OBFF Disabled ARIFwd- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1- EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest- Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit- Address: fee00258 Data: 0000 Capabilities: [90] Subsystem: Elitegroup Computer Systems 8 Series/C220 Series Chipset Family PCI Express Root Port Capabilities: [a0] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Kernel driver in use: pcieport Kernel modules: shpchp<div> </div></div></div><div> </div></div></div><div>pve-q35.cfg attempt to assign the pch device tree</div><div>[device "pcie.2"] driver = "pxb-pcie" multifunction = "on" addr = "1c.4" bus_nr = "0x08" [device "pci.21"] driver = "ioh3420" multifunction = "on" bus = "pcie.2" addr = "0.3" slot = "0x04" port = "5" chassis = "5"</div><div> </div><div>I've read the docs about the different approaches for virtual pcie ports, upstream and downstream switches and root ports, but the guest device manager reports the virtual ports only, still not getting the firewire board visible in the guest. </div><div> </div><div>I don't understand completely how can the virtual pcie ports be mapped to that 1c.4 [03-04] tree so I thought this would be the best place to get some light on the subject. </div><div> </div><div>the good part is that the host doesn't freeze when using the pve-q35.cfg approach to assign the device to the guest, but even if I get some close-numbered dev-ids assigned to the virtual pcie ports, they still don't relate to the physical device.</div><div> </div><div>this is the line about pch in dmesg:</div><div>Kernel: [ 0.080766] pci 0000:00:1c.4: Intel PCH root port ACS workaround enabled </div><div> </div><div>here is the dmesg log for the vfio-pci and pci-stub modules</div><div>[ 1.704898] vfio_pci: add [10de:1287[ffff:ffff]] class 0x000000/00000000 [ 1.704901] vfio_pci: add [10de:0e0f[ffff:ffff]] class 0x000000/00000000 [ 1.704902] vfio_pci: add [1019:7e44[ffff:ffff]] class 0x000000/00000000 [ 1.704904] vfio_pci: add [104c:8235[ffff:ffff]] class 0x000000/00000000</div><div>[ 1.701664] pci-stub: add 10DE:1287 sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000 [ 1.701674] pci-stub 0000:01:00.0: claimed by stub [ 1.701680] pci-stub: add 10DE:0E0F sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000 [ 1.701684] pci-stub 0000:01:00.1: claimed by stub [ 1.701686] pci-stub: add 1019:7E44 sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000 [ 1.701688] pci-stub: add 104C:8235 sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000 [ 1.701691] pci-stub 0000:04:00.0: claimed by stub</div></div>