<p dir="ltr">What is the driver version? Maybe it is a whole new way of blocking?</p>
<div class="gmail_quote">On Nov 2, 2015 3:11 PM, "Eddie Yen" <<a href="mailto:missile0407@gmail.com">missile0407@gmail.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">OK, but I still suggest that remove Hyper-V function tags in your XML.<div>Because we don't know about what new tricks inside the driver that NVIDIA wants to "surprise" us.</div><div><br></div><div>For me, my GTX980 works well by using upon edits. But I'm using 4820K which didn't need ACS patch and wo intel graphic.</div><div>So I'm not sure it may cause by patch or sth.</div></div><div class="gmail_extra"><br><div class="gmail_quote">2015-11-02 20:04 GMT+08:00 Georgios Kourachanis <span dir="ltr"><<a href="mailto:geo.kourachanis@gmail.com" target="_blank">geo.kourachanis@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Hello Eddie,<br>
<br>
Thanks for answering, though:<br>
<br>
What you suggest me to do, I've already done it with this way:<br>
<br>
<font color="#000000"><span><tt>/usr/local/bin/qemu-system-x86_64.hv</tt><tt>:</tt><tt><br>
</tt><tt>#!/bin/sh</tt><tt><br>
</tt><tt>exec /usr/bin/qemu-system-x86_64 `echo "\$@" | \</tt><tt><br>
</tt><tt>sed 's|hv_time|hv_time,hv_vendor_id=GoobyPLS|g'</tt></span><tt><br>
<br>
<br>
and by changing the emulator qemu to this line:<br>
<br>
<</tt></font><font color="#000000"><tt>emulator>/usr/local/bin/qemu-system-x86_64.hv</emulator></tt><tt><br>
</tt></font><br>
I'm just giving the ID "GoobyPLS" to the vendor. I'll try without
a vendor name to see if it changes anything.<br>
<br>
Also, I'm using the qemu git version "r41983.g3a958f5" so it
already contains the patch that helps us use the lines above.<div><div><br>
<br>
<br>
<br>
<br>
On 02/11/2015 03:53 πμ, Eddie Yen wrote:<br>
</div></div></div><div><div>
<blockquote type="cite">
<div dir="ltr">According from AW's blog:
<div>"<span style="font-size:13.1999998092651px;line-height:18.4799995422363px;font-family:Arial,Helvetica,sans-serif"> For
this step we again need to run </span><span>virsh edit</span><span style="font-size:13.1999998092651px;line-height:18.4799995422363px;font-family:Arial,Helvetica,sans-serif"> on
the VM. Within the</span><span><features></span><span style="font-size:13.1999998092651px;line-height:18.4799995422363px;font-family:Arial,Helvetica,sans-serif"> section,
remove everything between the </span><span><hyperv></span><span style="font-size:13.1999998092651px;line-height:18.4799995422363px;font-family:Arial,Helvetica,sans-serif"> tags,
including the tags themselves.0</span>"</div>
<div>and</div>
<div>"<span style="font-size:13.1999998092651px;line-height:18.4799995422363px;font-family:Arial,Helvetica,sans-serif">Additionally,
within the </span><span><clock></span><span style="font-size:13.1999998092651px;line-height:18.4799995422363px;font-family:Arial,Helvetica,sans-serif"> tag,
find the timer named </span><span>hypervclock</span><span style="font-size:13.1999998092651px;line-height:18.4799995422363px;font-family:Arial,Helvetica,sans-serif">,
remove the line containing this tag completely. Save and
exit the edit session.</span>"</div>
<div><br>
</div>
<div>I found that these still exist in your XML file, so try to
do this:</div>
<div><br>
</div>
<div>1. Remove these tags.</div>
<div>2. Re-compile QEMU and re-install it with this patch</div>
<div><a href="http://www.spinics.net/lists/kvm/msg121742.html" target="_blank">http://www.spinics.net/lists/kvm/msg121742.html</a><br>
</div>
<div>3. Add these tags between </devices> and
</domain></div>
<div><br>
</div>
<div><qemu:commandline></div>
<div> <qemu:arg value='-cpu'/></div>
<div> <qemu:arg
value='host,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,kvm=off,hv)vendor_id='/><br>
</div>
<div></qemu:commandline></div>
<div><br>
</div>
<div>I'm using GTX980, too. Before that, I got poor 3D
performance in Windows 10, after this patch and edition, I got
performance back.</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">2015-11-02 1:43 GMT+08:00 Georgios
Kourachanis <span dir="ltr"><<a href="mailto:geo.kourachanis@gmail.com" target="_blank">geo.kourachanis@gmail.com</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div>
<div bgcolor="#FFFFFF" text="#000000"> <tt>Hello </tt><tt>all,</tt><tt><br>
</tt><tt><br>
</tt><tt>I had been using Xen </tt><tt>with some AMD
GPUs </tt><tt>for almost 2 years till about June
2015, when I then found out that KVM and libvirt
could do the same stuff I was interested in with
nvidia GPUs, too. I needed the CUDA cores so I did
change to an ASUS GTX 980 Strix. But unfortunatelly,
I don't get any good performance output from it. On
native windows 7/10 installation it's a beast
though.</tt><tt><br>
</tt><tt>I also have an AMD R7 250 which works great
with KVM. But let's not mess with it.</tt><tt><br>
</tt><tt><br>
</tt><tt>Let me get to the point:</tt><tt><br>
</tt><tt><br>
</tt><tt>I have no problems as for the installation of
Windows or OVMF or passing-through or anything else.
The only problem is the GTX980's performance.</tt><tt><br>
</tt><tt>The performance had a significant boost when
I used the latest qemu branch with the hyper-v
trick, but still, not getting what many people seem
to claim in this mailing list "almost-native" (even
with nvidia GPUs).</tt><tt><br>
</tt><tt><br>
</tt><tt><br>
</tt><tt>Here is my system's specs:</tt><tt><br>
</tt><tt><br>
</tt><tt>Archlinux with 4.1.6-1-vfio (with the ACS
patch ALONE)</tt><tt><br>
</tt><tt>Intel Core i7</tt><tt> 3770 ( I use the igpu
for the archlinux)</tt><tt><br>
</tt><tt>24GiB RAM</tt><tt><br>
</tt><tt>ASUS GTX 980 Strix</tt><tt><br>
</tt><tt>Sapphire R7 250</tt><tt><br>
</tt>
<hr size="2" width="100%"><tt>lspci (only
pass-through'd stuff):</tt><tt><br>
</tt><tt><br>
</tt><tt>01:00.0 VGA compatible controller: NVIDIA
Corporation GM204 [GeForce GTX 980] (rev a1)</tt><tt><br>
</tt><tt> Subsystem: ASUSTeK Computer Inc.
Device 8518</tt><tt><br>
</tt><tt> Kernel driver in use: vfio-pci</tt><tt><br>
</tt><tt> Kernel modules: nouveau</tt><tt><br>
</tt><tt>01:00.1 Audio device: NVIDIA Corporation
GM204 High Definition Audio Controller (rev a1)</tt><tt><br>
</tt><tt> Subsystem: ASUSTeK Computer Inc.
Device 8518</tt><tt><br>
</tt><tt> Kernel driver in use: vfio-pci</tt><tt><br>
</tt><tt> Kernel modules: snd_hda_intel</tt><tt><br>
</tt><tt>02:00.0 VGA compatible controller: Advanced
Micro Devices, Inc. [AMD/ATI] Oland PRO [Radeon R7
240/340]</tt><tt><br>
</tt><tt> Subsystem: PC Partner Limited /
Sapphire Technology Device e266</tt><tt><br>
</tt><tt> Kernel modules: radeon</tt><tt><br>
</tt><tt>02:00.1 Audio device: Advanced Micro Devices,
Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio
[Radeon HD 7700/7800 Series]</tt><tt><br>
</tt><tt> Subsystem: PC Partner Limited /
Sapphire Technology Device aab0</tt><tt><br>
</tt><tt> Kernel driver in use: snd_hda_intel</tt><tt><br>
</tt><tt> Kernel modules: snd_hda_intel</tt><tt><br>
</tt><tt>08:00.0 USB controller: ASMedia Technology
Inc. ASM1042 SuperSpeed USB Host Controller</tt><tt><br>
</tt><tt> Subsystem: ASRock Incorporation
Motherboard</tt><tt><br>
</tt><tt> Kernel driver in use: vfio-pci</tt><tt><br>
</tt><tt> Kernel modules: xhci_pci</tt><tt><br>
</tt>
<hr size="2" width="100%"><tt>booting lines:</tt><tt><br>
</tt><tt><br>
</tt><tt>linux /boot/vmlinuz-linux-vfio
root=UUID=XXXX rw intel_iommu=on
pcie_acs_override=downstream isolcpus=2-3,6-7
nohz_full=2-3,6-7</tt><tt><br>
</tt><tt>initrd /boot/intel-ucode.img
/boot/initramfs-linux-vfio.img</tt><tt><br>
</tt>
<hr size="2" width="100%"><tt>/etc/fstab:</tt><code><br>
<br>
hugetlbfs /hugepages hugetlbfs defaults 0 0</code><tt><br>
</tt>
<hr size="2" width="100%"><tt>/etc/sysctl.d/40-hugepage.conf:</tt><tt><br>
</tt><tt><br>
</tt><tt>vm.nr_hugepages = 8000</tt><tt><br>
</tt>
<hr size="2" width="100%"><tt>/etc/modules-load.d/vfio.conf:</tt><tt><br>
</tt><tt><br>
</tt><tt>kvm</tt><tt><br>
</tt><tt>kvm-intel</tt><tt><br>
</tt><tt>vfio</tt><tt><br>
</tt><tt>vfio-pci</tt><tt><br>
</tt><tt>vfio_iommu_type1</tt><tt><br>
</tt><tt>vfio_virqfd</tt><tt><br>
</tt>
<hr size="2" width="100%"><tt>/etc/modprobe.d/kvm.conf:</tt><tt><br>
</tt><tt><br>
</tt><tt>options kvm ignore_msrs=1</tt><tt><br>
</tt>
<hr size="2" width="100%"><tt>/etc/modprobe.d/kvm-intel.conf:</tt><tt><br>
</tt><tt><br>
</tt><tt>options kvm-intel nested=1</tt><tt><br>
</tt>
<hr size="2" width="100%"><tt>/etc/modprobe.d/vfio_iommu_type1.conf:</tt><tt><br>
</tt><tt><br>
</tt><tt>options vfio_iommu_type1
allow_unsafe_interrupts=0</tt><tt><br>
</tt>
<hr size="2" width="100%"><tt>/etc/modprobe.d/vfio-pci.conf:</tt><tt><br>
</tt><tt><br>
</tt><tt>options vfio-pci
ids=10de:13c0,10de:0fbb,1002:6613,1002:aab0,1b21:1042</tt><tt><br>
</tt>
<hr size="2" width="100%"><tt><br>
</tt><tt>And the virsh xml:</tt><tt><br>
</tt><tt><br>
</tt><tt><domain type='kvm'></tt><tt><br>
</tt><tt> <name>windows_10</name></tt><tt><br>
</tt><tt>
<uuid>63045df8-c782-4cfd-abc7-a3598826ae83</uuid></tt><tt><br>
</tt><tt> <memory
unit='KiB'>6553600</memory></tt><tt><br>
</tt><tt> <currentMemory
unit='KiB'>6553600</currentMemory></tt><tt><br>
</tt><tt> <memoryBacking></tt><tt><br>
</tt><tt> <hugepages/></tt><tt><br>
</tt><tt> </memoryBacking></tt><tt><br>
</tt><tt> <vcpu
placement='static'>4</vcpu></tt><tt><br>
</tt><tt> <cputune></tt><tt><br>
</tt><tt> <vcpupin vcpu='0' cpuset='2'/></tt><tt><br>
</tt><tt> <vcpupin vcpu='1' cpuset='3'/></tt><tt><br>
</tt><tt> <vcpupin vcpu='2' cpuset='6'/></tt><tt><br>
</tt><tt> <vcpupin vcpu='3' cpuset='7'/></tt><tt><br>
</tt><tt> </cputune></tt><tt><br>
</tt><tt> <os></tt><tt><br>
</tt><tt> <type arch='x86_64'
machine='pc-i440fx-2.4'>hvm</type></tt><tt><br>
</tt><tt> <loader readonly='yes'
type='pflash'>/usr/local/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd</loader></tt><tt><br>
</tt><tt>
<nvram>/var/lib/libvirt/qemu/nvram/windows_nvidia_VARS.fd</nvram></tt><tt><br>
</tt><tt> </os></tt><tt><br>
</tt><tt> <features></tt><tt><br>
</tt><tt> <acpi/></tt><tt><br>
</tt><tt> <apic/></tt><tt><br>
</tt><tt> <pae/></tt><tt><br>
</tt><tt> <hyperv></tt><tt><br>
</tt><tt> <relaxed state='on'/></tt><tt><br>
</tt><tt> <vapic state='on'/></tt><tt><br>
</tt><tt> <spinlocks state='on'
retries='8191'/></tt><tt><br>
</tt><tt> </hyperv></tt><tt><br>
</tt><tt> <kvm></tt><tt><br>
</tt><tt> <hidden state='on'/></tt><tt><br>
</tt><tt> </kvm></tt><tt><br>
</tt><tt> <vmport state='off'/></tt><tt><br>
</tt><tt> </features></tt><tt><br>
</tt><tt> <cpu mode='host-passthrough'></tt><tt><br>
</tt><tt> <topology sockets='1' cores='4'
threads='1'/></tt><tt><br>
</tt><tt> </cpu></tt><tt><br>
</tt><tt> <clock offset='localtime'></tt><tt><br>
</tt><tt> <timer name='rtc'
tickpolicy='catchup'/></tt><tt><br>
</tt><tt> <timer name='pit'
tickpolicy='delay'/></tt><tt><br>
</tt><tt> <timer name='hpet' present='no'/></tt><tt><br>
</tt><tt> <timer name='hypervclock'
present='yes'/></tt><tt><br>
</tt><tt> </clock></tt><tt><br>
</tt><tt>
<on_poweroff>destroy</on_poweroff></tt><tt><br>
</tt><tt> <on_reboot>restart</on_reboot></tt><tt><br>
</tt><tt> <on_crash>restart</on_crash></tt><tt><br>
</tt><tt> <pm></tt><tt><br>
</tt><tt> <suspend-to-mem enabled='no'/></tt><tt><br>
</tt><tt> <suspend-to-disk enabled='no'/></tt><tt><br>
</tt><tt> </pm></tt><tt><br>
</tt><tt> <devices></tt><tt><br>
</tt><tt>
<emulator>/usr/local/bin/qemu-system-x86_64.hv</emulator></tt><tt><br>
</tt><tt> <disk type='block' device='disk'></tt><tt><br>
</tt><tt> <driver name='qemu' type='raw'
cache='none'/></tt><tt><br>
</tt><tt> <source
dev='/dev/mapper/vg_ssd-lv_kvm_NVIDIA'/></tt><tt><br>
</tt><tt> <target dev='sda' bus='scsi'/></tt><tt><br>
</tt><tt> <boot order='1'/></tt><tt><br>
</tt><tt> <address type='drive' controller='0'
bus='0' target='0' unit='0'/></tt><tt><br>
</tt><tt> </disk></tt><tt><br>
</tt><tt> <disk type='block' device='disk'></tt><tt><br>
</tt><tt> <driver name='qemu' type='raw'
cache='none'/></tt><tt><br>
</tt><tt> <source
dev='/dev/mapper/vg_raid5-lv_xen_ntfs_files'/></tt><tt><br>
</tt><tt> <target dev='sdb' bus='scsi'/></tt><tt><br>
</tt><tt> <address type='drive' controller='0'
bus='0' target='0' unit='1'/></tt><tt><br>
</tt><tt> </disk></tt><tt><br>
</tt><tt> <controller type='usb' index='0'></tt><tt><br>
</tt><tt> <address type='pci' domain='0x0000'
bus='0x00' slot='0x01' function='0x2'/></tt><tt><br>
</tt><tt> </controller></tt><tt><br>
</tt><tt> <controller type='pci' index='0'
model='pci-root'/></tt><tt><br>
</tt><tt> <controller type='scsi' index='0'
model='virtio-scsi'></tt><tt><br>
</tt><tt> <address type='pci' domain='0x0000'
bus='0x00' slot='0x06' function='0x0'/></tt><tt><br>
</tt><tt> </controller></tt><tt><br>
</tt><tt> <interface type='bridge'></tt><tt><br>
</tt><tt> <mac
address='52:54:00:e9:85:8f'/></tt><tt><br>
</tt><tt> <source bridge='xenbr0'/></tt><tt><br>
</tt><tt> <model type='e1000'/></tt><tt><br>
</tt><tt> <address type='pci' domain='0x0000'
bus='0x00' slot='0x03' function='0x0'/></tt><tt><br>
</tt><tt> </interface></tt><tt><br>
</tt><tt> <hostdev mode='subsystem' type='pci'
managed='yes'></tt><tt><br>
</tt><tt> <source></tt><tt><br>
</tt><tt> <address domain='0x0000'
bus='0x01' slot='0x00' function='0x0'/></tt><tt><br>
</tt><tt> </source></tt><tt><br>
</tt><tt> <address type='pci' domain='0x0000'
bus='0x00' slot='0x0a' function='0x0'
multifunction='on'/></tt><tt><br>
</tt><tt> </hostdev></tt><tt><br>
</tt><tt> <hostdev mode='subsystem' type='pci'
managed='yes'></tt><tt><br>
</tt><tt> <source></tt><tt><br>
</tt><tt> <address domain='0x0000'
bus='0x01' slot='0x00' function='0x1'/></tt><tt><br>
</tt><tt> </source></tt><tt><br>
</tt><tt> <address type='pci' domain='0x0000'
bus='0x00' slot='0x0a' function='0x1'/></tt><tt><br>
</tt><tt> </hostdev></tt><tt><br>
</tt><tt> <hostdev mode='subsystem' type='pci'
managed='yes'></tt><tt><br>
</tt><tt> <source></tt><tt><br>
</tt><tt> <address domain='0x0000'
bus='0x08' slot='0x00' function='0x0'/></tt><tt><br>
</tt><tt> </source></tt><tt><br>
</tt><tt> <address type='pci' domain='0x0000'
bus='0x00' slot='0x08' function='0x0'/></tt><tt><br>
</tt><tt> </hostdev></tt><tt><br>
</tt><tt> <memballoon model='virtio'></tt><tt><br>
</tt><tt> <address type='pci' domain='0x0000'
bus='0x00' slot='0x05' function='0x0'/></tt><tt><br>
</tt><tt> </memballoon></tt><tt><br>
</tt><tt> </devices></tt><tt><br>
</tt><tt></domain></tt><tt><br>
</tt>
<hr size="2" width="100%"><tt><br>
</tt><tt>/usr/local/bin/qemu-system-x86_64.hv</tt><tt>:</tt><tt><br>
</tt><tt>#!/bin/sh</tt><tt><br>
</tt><tt>exec /usr/bin/qemu-system-x86_64 `echo "\$@"
| \</tt><tt><br>
</tt><tt>sed
's|hv_time|hv_time,hv_vendor_id=GoobyPLS|g'</tt><tt><br>
</tt><tt><br>
</tt><tt><br>
</tt><tt><br>
</tt><tt>And some notes:</tt><tt><br>
</tt><tt><br>
</tt><tt>1) Using "</tt><tt><topology sockets='1'
cores='4' threads='1'/></tt><tt>" instead o</tt><tt>f
</tt><tt>"</tt><tt><topology sockets='1' cores='</tt><tt>2</tt><tt>'
threads='</tt><tt>2</tt><tt>'/></tt><tt>"
provided about 2% boost in GPU performance. No
change in RAM or CPU tests. I've tested with the
passmark</tt><tt>.</tt><tt><br>
</tt><tt><br>
</tt><tt>2) I tried using the emulatorpin method Alex
says on a mail here on </tt><tt>vfio-users</tt><tt>,
but I didn't notice any changed in GPU performance.
I didn't test it on the CPU side though.</tt><tt><br>
</tt><tt><br>
</tt><tt>3) The main problem of the performance lack
is that a specific game that I've been playing isn't
quite playable. That game has been mentioned before
here on the list, it's Tera (european version
(gameforge), although american version</tt><tt>(enmasse)
has exactly the same performance).</tt><tt><br>
</tt><tt><br>
</tt><tt>4) Every othe</tt><tt>r game I managed to
play is quite playable, though I haven't tested them
to see if they run on native speeds.</tt><tt><br>
</tt><tt><br>
</tt><tt><br>
</tt><tt>I'd really want some help on this matter, I
really want to make my server run this VM with the
nvidia GPU. I hate dual booting Windows </tt><tt>>_></tt><tt><br>
</tt><tt><br>
</tt><tt><br>
</tt><tt>Thank</tt><tt>s!</tt><tt><br>
</tt> </div>
</div>
</div>
<br>
_______________________________________________<br>
vfio-users mailing list<br>
<a href="mailto:vfio-users@redhat.com" target="_blank">vfio-users@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/vfio-users" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/vfio-users</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div></div></div>
</blockquote></div><br></div>
<br>_______________________________________________<br>
vfio-users mailing list<br>
<a href="mailto:vfio-users@redhat.com">vfio-users@redhat.com</a><br>
<a href="https://www.redhat.com/mailman/listinfo/vfio-users" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/vfio-users</a><br>
<br></blockquote></div>