Nice!! Thank you for the info.<div><br></div><div>Are you using zfs for your hypervisor root partition?</div><div><br></div><div><br></div><div><br><br>On Tuesday, September 15, 2015, Okky Hendriansyah <<a href="mailto:okky@nostratech.com">okky@nostratech.com</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">Hi Jouner, </div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">I’m doing exactly like you asked for. Here’s my full spec:</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><b>Processor</b>:</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">- Intel Core i7-4770</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><b>Motherboard</b>:</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">- ASRock Z87 Extreme6</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><b>Memory</b>:</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">- Patriot SLK DDR3 8 GB x 4</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><b>Graphics</b>:</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">- Intel HD Graphics 4600 (Hypervisor graphics)</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">- Gigabyte GeForce GTX 980 G1 Gaming (VM passthrough graphics)</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><b>Power Supply</b>:</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">- Seasonic M12II-750</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><b>SAS/SATA Controller</b>:</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">- IBM ServeRaid M1015</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><b>Storage</b>:</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">- Intel 530 120 GB (Arch Linux Hypervisor)</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">- Hitachi Deskstar 160 GB (currently as Windows 10 VM backup)</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">- Hitachi Deskstar 500 GB (Games attached to Windows 10 VM as raw disk)</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">- Hitachi Deskstar 2 TB (Multimedia attached to Windows 10 VM as raw disk)</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto">- WD Caviar Blue 1 TB x 8 (ZFS stripes of 2-way mirror, attached to IBM M1015)</div><div style="font-family:Helvetica,Arial;font-size:13px;color:rgba(0,0,0,1.0);margin:0px;line-height:auto"><br></div><div style="color:rgb(0,0,0);margin:0px"><span style="font-family:Helvetica,Arial;font-size:13px">I use <font face="Courier New">linux-vfio</font> kernel in conjunction with <font face="Courier New">zfs-git</font> (both are from AUR). Although I do not need Intel VGA Arbitration patch since I switched my Windows 10 VM to use OVMF, the </span><font face="Courier New">linux-vfio</font><span style="font-family:Helvetica,Arial;font-size:13px"> kernel is still needed since my system does not support ACS thus needs the PCIe ACS Override patch. Naturally the <font face="Courier New">zfs-git</font> AUR package is assuming to be loaded on vanilla <font face="Courier New">linux</font> kernel, so I need to edit the </span><font face="Courier New">PKGBUILD</font><span style="font-family:Helvetica,Arial;font-size:13px"> for </span><font face="Courier New">zfs-git</font> everytime it needs to be updated to not depend on <font face="Courier New">linux</font> kernel. So yes, I need to recompile everything if both and only both updates are available from AUR. But if you can do GPU passthrough with stock Arch Linux kernel, then you can use <font face="Courier New">zfs-git</font> package from AUR straight away.</div><div style="color:rgb(0,0,0);margin:0px"><br></div><div style="color:rgb(0,0,0);margin:0px">I allocated only 2 GB for ZFS ARC, so far it performs stable for my needs. Could be adjusted if needed. My Windows 10 VM disk is a raw image file (.img) based on Hitachi 160 GB and I put it on top of a ZFS dataset for increased bandwidth plus I can do ZFS snapshot/rollback everytime I needed to. Still curious of the performance if I put the Windows 10 VM in a ZVOL though. The IBM ServeRaid M1015 is crossflashed to LSI 9211-IT mode for dumb storage passthrough. </div><div style="color:rgb(0,0,0);margin:0px"><br></div><div style="color:rgb(0,0,0);margin:0px">Before my current graphics, I have successfully passthrough HIS R9 270X IceQ X2, Gigabyte GeForce GTX 770 Windforce, and ASUS GeForce GTX 780 DirectCU II to this machine, without reboot freeze whatsoever.</div><div style="color:rgb(0,0,0);margin:0px"><br></div><div style="color:rgb(0,0,0);margin:0px">Drooling looking at your spec, Sir.</div><div style="color:rgb(0,0,0);margin:0px"><br></div><div style="color:rgb(0,0,0);margin:0px">Best regards,</div> <div><font face="helvetica, arial">-- </font><br><span style="color:rgb(2,1,3)"><font face="Calibri"><b>Okky Hendriansyah</b></font></span><div><div><font face="Times New Roman"><span style="font-size:12px"></span></font></div></div></div> <br><p style="color:#000">On September 15, 2015 at 03:24:57, Blank Field (<a href="javascript:_e(%7B%7D,'cvml','ihatethisfield@gmail.com');" target="_blank">ihatethisfield@gmail.com</a>) wrote:</p> <blockquote type="cite"><span><div><div></div><div> <p dir="ltr">You've chosen some exceptionally good hardware build, E5 xeons shouldn't have ACS and other problems.<br> As for the kernel - nah, one or two configuration changes and recompiles and you're good to go.<br> Stock arch kernel doesn't even have vfio enabled, AFAIR.</p> <div class="gmail_quote">On Sep 14, 2015 7:03 PM, "Jouner Journer" <<a href="javascript:_e(%7B%7D,'cvml','journerforums@gmail.com');" target="_blank">journerforums@gmail.com</a>> wrote:<br type="attribution"> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div dir="ltr">Hi Everyone, <div><br></div> <div>I'm looking to build a box which can run ZFS on Linux in hypervisor, misc linux VMs, as well as a Windows vm for gaming.</div> <div><br></div> <div>Is anyone doing video passthrough as well as running ZFSoL? I've read ZFS requires specific kernel versions in Arch and am wondering if I'll run into compatibility issues.</div> <div><br></div> <div>Also, this is the hardware I've selected... are there any blatant problems that stick out? I've been toying with the idea of different processor (less cores, more speed), but figured more cores is better... how about Radeon R9 390X... is it too new to be tested/confirmed? </div> <div><br></div> <div>Given that it is xeon processor and motherboard supports VT-d/VT-x, motherboard and video card are uefi I'm hoping I will not run into too many issues with passthrough... at least hardware limited issues ;)</div> <div><br></div> <div>ASRock EPC612D8A-TB SSI ATX Server Motherboard Socket LGA 2011 R3<br></div> <div>Intel Xeon E5-2630 v3 Haswell-EP 2.4 GHz LGA 2011-3 85W BX80644E52630V3 Server Processor<br></div> <div>SAPPHIRE Radeon R9 390X 100381OCL 8GB 512-Bit GDDR5 PCI Express 3.0 2.2 Slot, ATX Tri-X OC Version (UEFI) Video Card<br></div> <div><br></div> <div>SeaSonic Platinum-1000 1000W PSU<br></div> <div>Intel 535 Series SSDSC2BW180H601 2.5" 180GB SATA III MLC SSD<br></div> <div>Intel 535 Series SSDSCKJW360H601 M.2 360GB SATA III MLC SSD<br></div> <div>Kingston 64GB (4 x 16GB) 288-Pin DDR4 SDRAM ECC Registered DDR4 2133 (PC4 17000) Server Memory Model KVR21R15D4K4/64<br></div> <div><br></div> <div><br></div> <div>Thanks in advance!</div> <div><br></div> <div><br></div> <div><br></div> <div><br></div> <div><br></div> <div><br></div> </div> <br> _______________________________________________<br> vfio-users mailing list<br> <a href="javascript:_e(%7B%7D,'cvml','vfio-users@redhat.com');" target="_blank">vfio-users@redhat.com</a><br> <a href="https://www.redhat.com/mailman/listinfo/vfio-users" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/vfio-users</a><br> <br></blockquote> </div> _______________________________________________ <br>vfio-users mailing list <br><a href="javascript:_e(%7B%7D,'cvml','vfio-users@redhat.com');" target="_blank">vfio-users@redhat.com</a> <br><a href="https://www.redhat.com/mailman/listinfo/vfio-users" target="_blank">https://www.redhat.com/mailman/listinfo/vfio-users</a> <br></div></div></span></blockquote></div></blockquote></div>