[vfio-users] VFIO + ZFS build

Tue Sep 15 05:16:41 UTC 2015

No, I do not setup ZFS on root for Arch Linux, I want the Hypervisor OS as simple as I can, in case there are issues in ZFS I can still boot Arch Linux straight away. Just a plain 2 partitions: /boot for EFI using FAT32 and / for everything else using Ext4. I kept my / partition purely for config, all my data and VM storages are at my ZFS array. Even my Downloads folder goes to ZFS.

Best regards,
Okky Hendriansyah

> On Sep 15, 2015, at 11:54, Jouner Journer <journerforums at gmail.com> wrote:
> 
> Nice!! Thank you for the info.
> 
> Are you using zfs for your hypervisor root partition?
> 
> 
> 
> 
>> On Tuesday, September 15, 2015, Okky Hendriansyah <okky at nostratech.com> wrote:
>> Hi Jouner, 
>> 
>> I’m doing exactly like you asked for. Here’s my full spec:
>> 
>> Processor:
>> - Intel Core i7-4770
>> 
>> Motherboard:
>> - ASRock Z87 Extreme6
>> 
>> Memory:
>> - Patriot SLK DDR3 8 GB x 4
>> 
>> Graphics:
>> - Intel HD Graphics 4600 (Hypervisor graphics)
>> - Gigabyte GeForce GTX 980 G1 Gaming (VM passthrough graphics)
>> 
>> Power Supply:
>> - Seasonic M12II-750
>> 
>> SAS/SATA Controller:
>> - IBM ServeRaid M1015
>> 
>> Storage:
>> - Intel 530 120 GB (Arch Linux Hypervisor)
>> - Hitachi Deskstar 160 GB (currently as Windows 10 VM backup)
>> - Hitachi Deskstar 500 GB (Games attached to Windows 10 VM as raw disk)
>> - Hitachi Deskstar 2 TB (Multimedia attached to Windows 10 VM as raw disk)
>> - WD Caviar Blue 1 TB x 8 (ZFS stripes of 2-way mirror, attached to IBM M1015)
>> 
>> I use linux-vfio kernel in conjunction with zfs-git (both are from AUR). Although I do not need Intel VGA Arbitration patch since I switched my Windows 10 VM to use OVMF, the linux-vfio kernel is still needed since my system does not support ACS thus needs the PCIe ACS Override patch. Naturally the zfs-git AUR package is assuming to be loaded on vanilla linux kernel, so I need to edit the PKGBUILD for zfs-git everytime it needs to be updated to not depend on linux kernel. So yes, I need to recompile everything if both and only both updates are available from AUR. But if you can do GPU passthrough with stock Arch Linux kernel, then you can use zfs-git package from AUR straight away.
>> 
>> I allocated only 2 GB for ZFS ARC, so far it performs stable for my needs. Could be adjusted if needed. My Windows 10 VM disk is a raw image file (.img) based on Hitachi 160 GB and I put it on top of a ZFS dataset for increased bandwidth plus I can do ZFS snapshot/rollback everytime I needed to. Still curious of the performance if I put the Windows 10 VM in a ZVOL though. The IBM ServeRaid M1015 is crossflashed to LSI 9211-IT mode for dumb storage passthrough. 
>> 
>> Before my current graphics, I have successfully passthrough HIS R9 270X IceQ X2, Gigabyte GeForce GTX 770 Windforce, and ASUS GeForce GTX 780 DirectCU II to this machine, without reboot freeze whatsoever.
>> 
>> Drooling looking at your spec, Sir.
>> 
>> Best regards,
>> -- 
>> Okky Hendriansyah
>> 
>>> On September 15, 2015 at 03:24:57, Blank Field (ihatethisfield at gmail.com) wrote:
>>> 
>>> You've chosen some exceptionally good hardware build, E5 xeons shouldn't have ACS and other problems.
>>> As for the kernel - nah, one or two configuration changes and recompiles and you're good to go.
>>> Stock arch kernel doesn't even have vfio enabled, AFAIR.
>>> 
>>>> On Sep 14, 2015 7:03 PM, "Jouner Journer" <journerforums at gmail.com> wrote:
>>>> Hi Everyone,
>>>> 
>>>> I'm looking to build a box which can run ZFS on Linux in hypervisor, misc linux VMs, as well as a Windows vm for gaming.
>>>> 
>>>> Is anyone doing video passthrough as well as running ZFSoL?  I've read ZFS requires specific kernel versions in Arch and am wondering if I'll run into compatibility issues.
>>>> 
>>>> Also, this is the hardware I've selected... are there any blatant problems that stick out?  I've been toying with the idea of different processor (less cores, more speed), but figured more cores is better...  how about Radeon R9 390X... is it too new to be tested/confirmed? 
>>>> 
>>>> Given that it is xeon processor and motherboard supports VT-d/VT-x, motherboard and video card are uefi I'm hoping I will not run into too many issues with passthrough... at least hardware limited issues ;)
>>>> 
>>>> ASRock EPC612D8A-TB SSI ATX Server Motherboard Socket LGA 2011 R3
>>>> Intel Xeon E5-2630 v3 Haswell-EP 2.4 GHz LGA 2011-3 85W BX80644E52630V3 Server Processor
>>>> SAPPHIRE Radeon R9 390X 100381OCL 8GB 512-Bit GDDR5 PCI Express 3.0 2.2 Slot, ATX Tri-X OC Version (UEFI) Video Card
>>>> 
>>>> SeaSonic Platinum-1000 1000W PSU
>>>> Intel 535 Series SSDSC2BW180H601 2.5" 180GB SATA III MLC SSD
>>>> Intel 535 Series SSDSCKJW360H601 M.2 360GB SATA III MLC SSD
>>>> Kingston 64GB (4 x 16GB) 288-Pin DDR4 SDRAM ECC Registered DDR4 2133 (PC4 17000) Server Memory Model KVR21R15D4K4/64
>>>> 
>>>> 
>>>> Thanks in advance!
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> vfio-users mailing list
>>>> vfio-users at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/vfio-users
>>> _______________________________________________ 
>>> vfio-users mailing list 
>>> vfio-users at redhat.com 
>>> https://www.redhat.com/mailman/listinfo/vfio-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20150915/1c7eca52/attachment.htm>