<div dir="ltr">I have GTX 970 and an intel buildin graphic I use for X11.<div>If you use ovmf you should at least get the efi startup screen on your nvidia output when your VM boots. Don't bother loading windows there, solve this first.</div><div>I have:</div><div><div>00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06)</div></div><div><div>BOOT_IMAGE=/kernel-genkernel-x86_64-4.5.1-gentoo root=UUID=8982a4f4-741e-4e77-9ac6-45422b169c4c ro rootfstype=ext4 init=/sbin/init intel_iommu=on pci-stub.ids=10de:13c2,10de:0fbb</div></div></div><br><div class="gmail_quote"><div dir="ltr">On Sat, May 7, 2016 at 9:43 AM Sheldon Palmer <<a href="mailto:slf_palmer@hotmail.com">slf_palmer@hotmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div dir="ltr">> > # cat /etc/modprobe.d/vfio.conf <br><div>> > options vfio-pci ids=<span style="background-color:rgb(255,255,255)">8086:0151,10de:17c8,10de:0fb0 <br><br>I deleted: 8086:0151</span></div></div></div><div><div dir="ltr"><div><span style="background-color:rgb(255,255,255)"><br><br>> Do NOT add the root port here, it needs to be bound to the default<br>> driver which will be pcieport. If the bridge is not enabled we can't<br>> talk to the card behind it. I'm not sure if this is actually taking<br>> affect though since vfio-pci in your kernel shouldn't bind to it.<br>> <br>> > options vfio-pci disable_vga=1<br>> > <br>> > # cat /etc/modprobe.d/kvm.conf <br>> > # if vfio-pci was built as a module ( default on arch & ubuntu )<br>> > options vfio_iommu_type1 allow_unsafe_interrupts=1 <br><br></span></div></div></div><div><div dir="ltr"><div><span style="background-color:rgb(255,255,255)">And got rid of this: options vfio_iommu_type1 allow_unsafe_interrupts=1 <br></span></div></div></div><div><div dir="ltr"><div><span style="background-color:rgb(255,255,255)"><br><br><br>> This is IGD going whacko, is this just a result of enabling the IOMMU<br>> or does it have some relation to the VM starting? Have you tried<br>> intel_iommu=on,igfx_off? It might be a bug in your host firmware not<br>> enabling RMRR entries for stolen memory. iommu=pt may also resolve it.<br><br></span></div></div></div><div><div dir="ltr"><div><span style="background-color:rgb(255,255,255)">Adding "igfx_off" got rid of the DMA write errors<br><br><br>Removed all of this:</span></div></div></div><div><div dir="ltr"><div><span style="background-color:rgb(255,255,255)"><br>> > <qemu:commandline><br>> > <qemu:arg value='-drive'/><br>> > <qemu:arg value='if=pflash,format=raw,readonly,file=/usr/share/edk2.git/ovmf-x64/OVMF-pure-efi.fd'/><br>> > </qemu:commandline><br></span></div></div></div><div><div dir="ltr"><div><br>> My only suspicion is the misuse of that root port by trying to bind it<br>> to vfio-pci. Undo that, if it still doesn't work send 'sudo lspci<br>> -vvv' for 00:01.0, and 01:<br><br><br></div></div></div><div><div dir="ltr"><div>Still just a flicker then the monitor turns off so here is sudo lspci -vvv<br><br>00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port (rev 09) (prog-if 00 [Normal decode])<br> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+<br> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-<br> Latency: 0, Cache Line Size: 64 bytes<br> Interrupt: pin A routed to IRQ 26<br> Bus: primary=00, secondary=01, subordinate=01, sec-latency=0<br> I/O behind bridge: 0000e000-0000efff<br> Memory behind bridge: f6000000-f70fffff<br> Prefetchable memory behind bridge: 00000000e0000000-00000000f1ffffff<br> Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-<br> BridgeCtl: Parity- SERR- NoISA- VGA+ MAbort- >Reset- FastB2B-<br> PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-<br> Capabilities: [88] Subsystem: ASUSTeK Computer Inc. P8H77-I Motherboard<br> Capabilities: [80] Power Management version 3<br> Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)<br> Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-<br> Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-<br> Address: feeff00c Data: 41a1<br> Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00<br> DevCap: MaxPayload 256 bytes, PhantFunc 0<br> ExtTag- RBE+<br> DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-<br> RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-<br> MaxPayload 256 bytes, MaxReadReq 128 bytes<br> DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-<br> LnkCap: Port #2, Speed 8GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <256ns, L1 <8us<br> ClockPM- Surprise- LLActRep- BwNot+ ASPMOptComp+<br> LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+<br> ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-<br> LnkSta: Speed 8GT/s, Width x16, TrErr- Train+ SlotClk+ DLActive- BWMgmt+ ABWMgmt+<br> SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-<br> Slot #1, PowerLimit 75.000W; Interlock- NoCompl+<br> SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-<br> Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-<br> SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-<br> Changed: MRL- PresDet+ LinkState-<br> RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-<br> RootCap: CRSVisible-<br> RootSta: PME ReqID 0000, PMEStatus- PMEPending-<br> DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported ARIFwd-<br> DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-<br> LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-<br> Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-<br> Compliance De-emphasis: -6dB<br> LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+, EqualizationPhase1+<br> EqualizationPhase2+, EqualizationPhase3+, LinkEqualizationRequest-<br> Capabilities: [100 v1] Virtual Channel<br> Caps: LPEVC=0 RefClk=100ns PATEntryBits=1<br> Arb: Fixed- WRR32- WRR64- WRR128-<br> Ctrl: ArbSelect=Fixed<br> Status: InProgress-<br> VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-<br> Arb: Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-<br> Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff<br> Status: NegoPending- InProgress-<br> Capabilities: [140 v1] Root Complex Link<br> Desc: PortNumber=02 ComponentID=01 EltType=Config<br> Link0: Desc: TargetPort=00 TargetComponent=01 AssocRCRB- LinkType=MemMapped LinkValid+<br> Addr: 00000000fed19000<br> Capabilities: [d94 v1] #19<br> Kernel driver in use: pcieport<br><br><br><br>01:00.0 VGA compatible controller: NVIDIA Corporation GM200 [GeForce GTX 980 Ti] (rev a1) (prog-if 00 [VGA controller])<br> Subsystem: ASUSTeK Computer Inc. GM200 [GeForce GTX 980 Ti]<br> Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-<br> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-<br> Interrupt: pin A routed to IRQ 16<br> Region 0: Memory at f6000000 (32-bit, non-prefetchable) [size=16M]<br> Region 1: Memory at e0000000 (64-bit, prefetchable) [size=256M]<br> Region 3: Memory at f0000000 (64-bit, prefetchable) [size=32M]<br> Region 5: I/O ports at e000 [size=128]<br> Expansion ROM at f7000000 [disabled] [size=512K]<br> Capabilities: [60] Power Management version 3<br> Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)<br> Status: D3 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-<br> Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+<br> Address: 0000000000000000 Data: 0000<br> Capabilities: [78] Express (v2) Legacy Endpoint, MSI 00<br> DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 <64us<br> ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-<br> DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-<br> RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+<br> MaxPayload 256 bytes, MaxReadReq 512 bytes<br> DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-<br> LnkCap: Port #0, Speed 8GT/s, Width x16, ASPM not supported, Exit Latency L0s <1us, L1 <4us<br> ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+<br> LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+<br> ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-<br> LnkSta: Speed 8GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-<br> DevCap2: Completion Timeout: Range AB, TimeoutDis+, LTR+, OBFF Via message<br> DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled<br> LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-<br> Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-<br> Compliance De-emphasis: -6dB<br> LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+, EqualizationPhase1+<br> EqualizationPhase2+, EqualizationPhase3+, LinkEqualizationRequest-<br> Capabilities: [100 v1] Virtual Channel<br> Caps: LPEVC=0 RefClk=100ns PATEntryBits=1<br> Arb: Fixed- WRR32- WRR64- WRR128-<br> Ctrl: ArbSelect=Fixed<br> Status: InProgress-<br> VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-<br> Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-<br> Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=01<br> Status: NegoPending- InProgress-<br> Capabilities: [250 v1] Latency Tolerance Reporting<br> Max snoop latency: 0ns<br> Max no snoop latency: 0ns<br> Capabilities: [258 v1] L1 PM Substates<br> L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+<br> PortCommonModeRestoreTime=255us PortTPowerOnTime=10us<br> Capabilities: [128 v1] Power Budgeting <?><br> Capabilities: [420 v2] Advanced Error Reporting<br> UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-<br> UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-<br> UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-<br> CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-<br> CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+<br> AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-<br> Capabilities: [600 v1] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?><br> Capabilities: [900 v1] #19<br> Kernel driver in use: vfio-pci<br> Kernel modules: nvidia<br><br><br><br>01:00.1 Audio device: NVIDIA Corporation Device 0fb0 (rev a1)<br> Subsystem: ASUSTeK Computer Inc. Device 8548<br> Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-<br> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-<br> Interrupt: pin B routed to IRQ 17<br> Region 0: Memory at f7080000 (32-bit, non-prefetchable) [size=16K]<br> Capabilities: [60] Power Management version 3<br> Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)<br> Status: D3 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-<br> Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+<br> Address: 0000000000000000 Data: 0000<br> Capabilities: [78] Express (v2) Endpoint, MSI 00<br> DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 <64us<br> ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-<br> DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-<br> RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+<br> MaxPayload 256 bytes, MaxReadReq 512 bytes<br> DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-<br> LnkCap: Port #0, Speed 8GT/s, Width x16, ASPM not supported, Exit Latency L0s <1us, L1 <4us<br> ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+<br> LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+<br> ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-<br> LnkSta: Speed 8GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-<br> DevCap2: Completion Timeout: Range AB, TimeoutDis+, LTR+, OBFF Via message<br> DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled<br> LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-<br> EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-<br> Capabilities: [100 v2] Advanced Error Reporting<br> UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-<br> UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-<br> UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-<br> CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-<br> CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+<br> AERCap: First Error Pointer: 00, GenCap- CGenEn- ChkCap- ChkEn-<br> Kernel driver in use: vfio-pci<br> Kernel modules: snd_hda_intel<br><br><br>Appreciate it Alex, thank you!<br><br><br><br></div> </div></div>
_______________________________________________<br>
vfio-users mailing list<br>
<a href="mailto:vfio-users@redhat.com" target="_blank">vfio-users@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/vfio-users" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/vfio-users</a><br>
</blockquote></div>