<div dir="ltr">You don't happen to have more PCIE slots on that MB that you can also try? I had a similar issue on the first slot and I believe I resolved it by adding "nomodeset vga=normal vesafb:off" to the kernel cmd line in GRUB config. I currently have it as<div>GRUB_CMDLINE_LINUX="nomodeset vga=normal video=vesafb:off"</div><div>and the boot is happening in the default text mode. Before I changed it the boot sequence was somehow affecting that card performance. I am not sure which one of these 3 settings helped, could be just one but I added them all together. Blacklisting your card driver was also necessary but I believe you have it done already.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Oct 3, 2015 at 1:37 PM, Pedro Henrique Lara Campos <span dir="ltr"><<a href="mailto:pedro.laracampos@gmail.com" target="_blank">pedro.laracampos@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div>I didn't gave up hope yet, If it was a pci-e 3.0 to pci-e 2.0 I would be using the second slot without problems, but, in this case, it's from a pci-e 2.0 x16 to a x4. And yes, MSI was also reducing my performance, but if there is a way for me to guarantee 100% of the available GPU power available, why would I give up...<br><br></div>I decided to change the GPUs again, log everything I could as well log when it doesn't work.<br></div>I've resumed them all here: <a href="https://gist.github.com/PedroHLC/5c54785a00efa3f08154" target="_blank">https://gist.github.com/PedroHLC/5c54785a00efa3f08154</a><br></div>As well as I added relevant settings and some hardware specs.<br><br></div>I've observed some things, while on PCIEX4 bus (secondary), I do not need a ROM, and if I pass one, even a downloaded one or one I ripped myself, It wouldn't work.<br></div>QEMU only throws ROM errors when on PCIEX16 bus (primary), as well it seems to be missing this on dmesg: "vfio-pci 0000:0?:00.0: enabling device (???? -> ????)"<br><br></div>So as, VFIO isn't enabling it, there is still something holding it up? If it was, shouldn't it appear on /proc/iomem? Is there a way to vfio verbose more intel?<div><div class="h5"><br><div class="gmail_extra"><br><div class="gmail_quote">2015-10-02 18:26 GMT-03:00 Blank Field <span dir="ltr"><<a href="mailto:ihatethisfield@gmail.com" target="_blank">ihatethisfield@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">To answer your last question you must examine your dmesg.</p> <p dir="ltr">Don't worry, on a system level it is possible to eject any device in windows. I've seen a hardware platform that could eject one of cpu's cores and system boot drive too. As in Linux, when you do that either nothing happens or nothing good happens.</p> <p dir="ltr">I personally would test if your GPU really needs the full pci-e 3.0 bandwidth. I mean, when you got it running, maybe it lags due to msi, cpu latency issues or something else?<br> Because HDMI audio stuttering problem points me directly to alex's blog post about message signalled interrupts.<br> And yeah, if you could switch the primary GPU the gigabyte way - that would solve your initial problem. So consider updating your firmware...</p><div><div> <div style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div>Both Seabios and OVMF.<br>Turn on/off the machine? I did tried restarting it sometimes...<br></div>Attaching an QXL video I can see NVIDIA driver with the classic 43 error, but I don't thing of it being qemu fault as when vga is in slot2 this doesn't happen.<br></div>If I disable nvidia device and reenable it, it goes back to error 43.<br></div>Weird observation, I can eject the GTX780, and the SCSI drivers in Windows eject tray icon....<br></div>For one last test, I uninstalled the drive in devmgmt, turned off the machine, turned it on without QXL, waited a long time to IDK, maybe Windows to reinstall the GPU, called "system_powerdown", ran it again in hope, but none.<br><br></div>Also, should my GPU still be the "target" of vga_arbiter?<br></div>If I'm using 'options vfio-pci ids=' should I also try pci-stub?<br></div><div class="gmail_extra"><br><div class="gmail_quote">2015-10-02 13:47 GMT-03:00 Blank Field <span dir="ltr"><<a href="mailto:ihatethisfield@gmail.com" target="_blank">ihatethisfield@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">Seabios? OVMF? Have you tried turining it on and off?^W^W^W^W^W reinstalling windows guest drivers? Attaching an emulated QXL device to see devmgmt.msc?</p><div><div> <div class="gmail_quote">On Oct 2, 2015 7:43 PM, "Pedro Henrique Lara Campos" <<a href="mailto:pedro.laracampos@gmail.com" target="_blank">pedro.laracampos@gmail.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Yep, you both we're right, uvesafb was using GPU, now nothing besides vfio is using it, yet I have no video, but there are no more mmap unsupported errors, with a romcard there is no ROM error, there are also no errors in AMD-Vi, qemu, IOMMU. Windows still loads fine, I can turn it off through monitor, every qemu setting is identical it was when GTX was in slot2 (in this slot I could see from OVMF bios until machine goes off) with exception of host pci addr but in slot1 it doesn't seems to want to start at all...<br><div><div class="gmail_extra"><br><div class="gmail_quote">2015-10-02 10:27 GMT-03:00 Blank Field <span dir="ltr"><<a href="mailto:ihatethisfield@gmail.com" target="_blank">ihatethisfield@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">Do cat /proc/iomem to find out who is using your GPU. I predict uvesafb or something like that. Disable or uninstall this.<br> Also, don't you have pci-e 3.0 onboard?</p> <div class="gmail_quote"><div><div>On Oct 2, 2015 4:22 PM, "Pedro Henrique Lara Campos" <<a href="mailto:pedro.laracampos@gmail.com" target="_blank">pedro.laracampos@gmail.com</a>> wrote:<br type="attribution"></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr"><div><div><div><div> Hello masters of vfio. I've been playing with vfio for 3 weeks now, and here is a resume of what i got: When I put my loved GTX780 on the second pci-e slot of my GA-970A-DS3P everything works on Windows (guest) while my Linux (host) stays comfortable with a Radeon HD 6670.<br></div>But, as I wouldn't be writing this email if nothing was wrong, my second pci-e slot is "PCIEX4" only, I believe that's because my FPS goes down and HDMI audio keeps crackin. I had to change the slots, my bios doesn't have an option to choose which GPU to use, so it uses the GTX, GRUB does it too, then gpu gets binded by modprobe and I have no video until XOrg starts on Radeon, if I start qemu, I've get:<br> "vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900"<br> "BAR 3 mmap unsupported. Performance may be slow"<br> "vfio-pci 0000:01:00.0: Invalid ROM contents"<br></div>No video on GTX, Windows boots normally, I can even hear notifications sounds, I can turn it off through "system_powerdown".<br></div>Tried using a romfile, it only makes the third message goes away, no other thing changes...<br></div></div> <br></div></div>_______________________________________________<br> vfio-users mailing list<br> <a href="mailto:vfio-users@redhat.com" target="_blank">vfio-users@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/vfio-users" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/vfio-users</a><br> <br></blockquote></div> </blockquote></div><br></div></div></div> </blockquote></div> </div></div></blockquote></div><br></div> </div> </div></div></blockquote></div><br></div></div></div></div> <br>_______________________________________________<br> vfio-users mailing list<br> <a href="mailto:vfio-users@redhat.com">vfio-users@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/vfio-users" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/vfio-users</a><br> <br></blockquote></div><br></div>