<!DOCTYPE html><html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8" /></head><body><div data-html-editor-font-wrapper="true" style="font-family: arial, sans-serif; font-size: 13px;">This sounds really a lot like that old kernel MTRR problem I described in my previous post, though it resulted in boot times on the order of tens of minutes. Which kernel version do you have?<br><br>Here is the information you requested:<br><br>Yes, I'm using libvirt. The set of qemu command-line arguments it produces is as follows:<br><br>-name guest=win10,debug-threads=on<br>-S<br>-object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-win10/master-key.aes<br>-machine pc-i440fx-2.5,accel=kvm,usb=off,vmport=off,dump-guest-core=off<br>-cpu host,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_vendor_id=KeenlyKVM,kvm=off<br>-drive file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on<br>-drive file=/var/lib/libvirt/qemu/nvram/win10_VARS.fd,if=pflash,format=raw,unit=1<br>-m 12288<br>-mem-prealloc<br>-mem-path /dev/hugepages/libvirt/qemu/3-win10<br>-realtime mlock=off<br>-smp 6,sockets=1,cores=3,threads=2<br>-object iothread,id=iothread1<br>-object iothread,id=iothread2<br>-uuid somelong-uuid-here<br>-display none<br>-no-user-config<br>-nodefaults<br>-chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-3-win10/monitor.sock,server,nowait<br>-mon chardev=charmonitor,id=monitor,mode=control<br>-rtc base=localtime,driftfix=slew<br>-global kvm-pit.lost_tick_policy=delay<br>-no-hpet<br>-no-shutdown<br>-global PIIX4_PM.disable_s3=1<br>-global PIIX4_PM.disable_s4=1<br>-boot strict=on<br>-device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7<br>-device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6<br>-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1<br>-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2<br>-drive file=/dev/sdc,format=raw,if=none,id=drive-virtio-disk0,cache=none,aio=native<br>-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1<br>-netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=26<br>-device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:11:22:33:44:55,bus=pci.0,addr=0x3<br>-device vfio-pci,host=05:00.0,id=hostdev0,bus=pci.0,addr=0x4,rombar=1<br>-device vfio-pci,host=05:00.1,id=hostdev1,bus=pci.0,addr=0x5,rombar=1<br>-device vfio-pci,host=00:1b.0,id=hostdev2,bus=pci.0,addr=0x2<br>-device usb-host,hostbus=1,hostaddr=5,id=hostdev3,bus=usb.0,port=1<br>-device usb-host,hostbus=1,hostaddr=6,id=hostdev4,bus=usb.0,port=2<br>-device usb-host,hostbus=1,hostaddr=7,id=hostdev5,bus=usb.0,port=3<br>-msg timestamp=on<br><br>Besides the rombar=1 for the passed-through GPU functions and mlock=off in the realtime parameter, I don't see much difference here.<br><br>The kernel boot arguments are similar to yours:<br><br>intel_iommu=on iommu=pt<br>hugepagesz=1G default_hugepagesz=1G hugepages=12<br>isolcpus=3,4,5,9,10,11 nohz_full=3,4,5,9,10,11 rcu_nocbs=3,4,5,9,10,11<br>pci-stub.ids=10de:13c2,10de:0fbb,8086:8d20<br>nvidia-drm.modeset=1<br><br>8086:8d20 is the on-board audio controller at 00:1b.0.<br><br>Hope that helps.<br><br>Cheers,<br>Hristo<br><br>25. Juli 2017 02:22, "John Koelndorfer" <<a target="_blank" tabindex="-1" href="mailto:%22John%20Koelndorfer%22%20<jkoelndorfer@gmail.com>">jkoelndorfer@gmail.com</a>> schrieb:<br> <blockquote><div><div> <div dir="ltr"> <div> <div>An update:<br> </div>The slow boot does not occur if I remove host devices from the virtual machine (without any other configuration changes). It doesn't matter whether GPU, USB, or both are passed in. Any device being passed in at all triggers the problem, so it's something related to the physical device passthrough I think.<br> </div> <div>An older version of OVMF did not help, nor did booting a non-Windows OS. I tried two alternate versions of OVMF from here: <a rel="external nofollow noopener noreferrer" target="_blank" tabindex="-1" href="https://www.rpmfind.net/linux/rpm2html/search.php?query=edk2-ovmf">https://www.rpmfind.net/linux/rpm2html/search.php?query=edk2-ovmf</a>. One from 2016 November, one from 2016 April. I initially did my last VFIO setup in 2016 July.<br> </div> <div>I also tried the pc-i440fx machine, per Hristo's suggestion (though I tried 2.6, which was what my last configuration used).<br> </div> <div>Hristo, it seems your setup is very similar to mine, so I have a few questions for you:<br><br>* Are you using libvirt?<br>* Could you send the qemu arguments for your VM?</div> <div>* What are your kernel boot parameters?</div> <div></div> <div>My suspicion now is that libvirt is doing some extra configuration that I'm not. I looked at my old libvirt XML file and nothing is jumping out at me, so maybe it's a hardcoded default behavior.<br> </div> <div>Thanks for all the suggestions so far, folks.</div> </div> <div> <div>On Mon, Jul 24, 2017 at 4:03 PM, Hagbard Celine <span dir="ltr"><<a target="_blank" rel="external nofollow noopener noreferrer" tabindex="-1" href="mailto:hagbardcelin@gmail.com">hagbardcelin@gmail.com</a>></span> wrote:<br><br> <blockquote style="margin: 0 0 0 .8ex;border-left: 1px #ccc solid;padding-left: 1ex">Hi, just registered to the list to share my experience on this;<br><br>I've been getting my OVMF builds from <a rel="external nofollow noopener noreferrer" target="_blank" tabindex="-1" href="http://kraxel.org">kraxel.org</a> since<br>edk2.git-ovmf-x64-0-20150804.<wbr></wbr>b1143.g8ca1489.noarch.rpm and was<br>regularly updating when new builds came.<br>Somwhere around edk2.git-ovmf-x64-0-20151221.<wbr></wbr>b1390.g5ba9f06.noarch.rpm<br>boot with large amounts of memory got slower.<br>And around edk2.git-ovmf-x64-0-20160324.<wbr></wbr>b1634.g3decba3.noarch.rpm the<br>EFI part of boot could last about 15min with 16GB assigned to VM out<br>of a total of 32GB. Lovering the assigned memory for VM to below 8GB<br>resulted in normal boot times.<br><br>PS. The version numbers are approximations, my recollection is not exact.<div><div> <br>______________________________<wbr></wbr>_________________<br>vfio-users mailing list<br><a rel="external nofollow noopener noreferrer" target="_blank" tabindex="-1" href="mailto:vfio-users@redhat.com">vfio-users@redhat.com</a><br><a rel="external nofollow noopener noreferrer" target="_blank" tabindex="-1" href="https://www.redhat.com/mailman/listinfo/vfio-users">https://www.redhat.com/<wbr></wbr>mailman/listinfo/vfio-users</a> </div></div> </blockquote> </div> </div> </div></div></blockquote> </div></body></html>