<div dir="ltr"><span style="font-size:12.8px">"options kvm ignore_msrs=1"<br></span><br>If you guys don't need it that is because you don't game.<br>Launch metal gear solid phantom pain on a windows guest without this option and your virtual machine will crash, period.<div>This is mandatory for every gamer who plays recent games.</div></div><div class="gmail_extra"><br><div class="gmail_quote">2016-04-12 23:24 GMT+02:00 Alex Williamson <span dir="ltr"><<a href="mailto:alex.l.williamson@gmail.com" target="_blank">alex.l.williamson@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><span class="">On Tue, Apr 12, 2016 at 2:30 PM, Bronek Kozicki <span dir="ltr"><<a href="mailto:brok@spamcop.net" target="_blank">brok@spamcop.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span>On 12/04/2016 20:36, Nicolas Roy-Renaud wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I've already rewritten the first two sections ("Prerequisites" and<br>
"Setting up IOMMU"), and the rest of the article should essentially<br>
follow the same basic structure and style. Replies here or on the wiki's<br>
discussion page would be much appreciated.<br>
</blockquote>
<br></span>
Hope Alex can clear any misconception I am about to present below<br>
<br>
<br>
1. I am using option iommu=pt, under impression this is expected to improve performance (my CPU is Xeon IvyBridge)<br></blockquote><div><br></div></span><div>It only affects the performance of host devices. There's less latency if host DMAs don't do dynamic mappings through the IOMMU, also less isolation between drivers.</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
2. does PCI bridge have to be in a separate IOMMU group than passed-through device?<br></blockquote><div><br></div></span><div>No. Blank is mostly correct on this, newer kernel remove the pcieport driver test and presumes any driver attached to a bridge device is ok.</div><div><div class="h5"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
3. would be nice to provide hints for headless host. FWIW, I use combination of<br>
3.1. kernel options:<br>
console=ttyS0,115200N8R nomodest video=vesa:off video=efifb:off vga=normal<br>
3.2.following line in /etc/modprobe.d/vfio.conf:<br>
options vfio-pci disable_vga=1<br>
3.3. large list of blacklisted modules (all framebuffers and nvidia and AMD drivers) in /etc/modprobe.d/blacklist.conf:<br>
# This host is headless, prevent any modules from attaching to video hardware<br>
# NVIDIA<br>
blacklist nouveau<br>
blacklist nvidia<br>
# AMD<br>
blacklist radeon<br>
blacklist amdgpu<br>
blacklist amdkfd<br>
blacklist fglrx<br>
# HDMI sound on a GPU<br>
blacklist snd_hda_intel<br>
# Framebuffers (ALL of them)<br>
blacklist vesafb<br>
blacklist aty128fb<br>
blacklist atyfb<br>
blacklist radeonfb<br>
blacklist cirrusfb<br>
blacklist cyber2000fb<br>
blacklist cyblafb<br>
blacklist gx1fb<br>
blacklist hgafb<br>
blacklist i810fb<br>
blacklist intelfb<br>
blacklist kyrofb<br>
blacklist lxfb<br>
blacklist matroxfb_base<br>
blacklist neofb<br>
blacklist nvidiafb<br>
blacklist pm2fb<br>
blacklist rivafb<br>
blacklist s1d13xxxfb<br>
blacklist savagefb<br>
blacklist sisfb<br>
blacklist sstfb<br>
blacklist tdfxfb<br>
blacklist tridentfb<br>
blacklist vfb<br>
blacklist viafb<br>
blacklist vt8623fb<br>
blacklist udlfb<br></blockquote><div><br></div></div></div><div>I suspect that blacklisting framebuffer drivers doesn't actually do anything.</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
4. ignore_msrs=1 also helps running Linux guests<br></blockquote><div><br></div></span><div>Never been needed in my experience.</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
5. do not use qemu:arg for binding host device to guest, here is example how to do it properly:<br>
<hostdev mode='subsystem' type='pci' managed='yes'><br>
<driver name='vfio'/><br>
<source><br>
<address domain='0x0000' bus='0x82' slot='0x00' function='0x0'/><br>
</source><br>
<address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/><br>
</hostdev><br>
5.1. for nVidia Quadro, add just below </source ...><br>
<rom bar='off'/><br></blockquote><div><br></div></span><div>Shouldn't be necessary to hide the ROM, Quadro should never be the primary graphics device and thus its ROM should never get executed.</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
6. if guest is started from BIOS rather than UEFI, keep the above <hostdev ...> but replace emulator with a script, e.g.<br>
# virsh dumpxml gdynia-vfio1 | grep emulator<br>
<emulator>/usr/bin/<a href="http://qemu-system-x86_64.xvga.sh" rel="noreferrer" target="_blank">qemu-system-x86_64.xvga.sh</a></emulator><br>
# cat /usr/bin/<a href="http://qemu-system-x86_64.xvga.sh" rel="noreferrer" target="_blank">qemu-system-x86_64.xvga.sh</a><br>
#!/bin/sh<br>
exec nice --adjustment=-5 /usr/bin/qemu-system-x86_64 `echo "$@" | \<br>
sed 's/-device vfio-pci,host=82:00.0/-device vfio-pci,host=82:00.0,x-vga=on/g' | \<br>
sed 's/-device vfio-pci,host=03:00.0/-device vfio-pci,host=03:00.0,x-vga=on/g'`<br>
<br>
<br>
7. performance optimizations<br>
7.1. use huge pages<br>
7.2. use isolcpus<br>
7.3. use vCPU pinnig<br>
7.4. use virtio-scsi with multiple queues (depending on number of available CPUs, after removing these dedicated to guest)<br>
7.5. use multiple queues for virtio-net<br>
7.6. for Linux guests, use P9 for mounting host filesystems in guest</blockquote><div><br></div></span><div>There are numerous ways to do this, that's one. Hard to make any universal recommendation there, NFS, sshfs, SMB are also options.</div></div></div></div>
<br>_______________________________________________<br>
vfio-users mailing list<br>
<a href="mailto:vfio-users@redhat.com">vfio-users@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/vfio-users" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/vfio-users</a><br>
<br></blockquote></div><br></div>