[vfio-users] VFIO + ZFS build

[globalgorrilla at fastmail.fm]

I'm running ZFSoL on 4.1.6 and using an R9 290 and Quadro 2000.

Both cards run well (both reset and are fully supported in the VM with 
HyperV extensions).

ZFS is memory hungry and I could use more than 32 GB RAM.

ZFSoL as of a couple days supports Linux 4.2 but my QEMU/OVMF VMs don't 
work with 4.2. I've heard some here say it might be OVMF.

My biggest issue once I figured out the quirks for my MB has been that 
of IRQ sharing on my Z97 board (Z97-WS).

The Quadro especially does not like to share an IRQ (with any other 
device). The Radeons seem less drastically affected but some 
combinations of Radeons don't work with shared IRQs. That seems to be 
something with the cards and not specific to vfio or Linux. ASUS 
documents their IRQ combinations but in reality once you disable some 
onboard devices it's a crapshoot as to what IRQs get assigned. Your 
board's manual doesn't mention IRQ assignments and I don't have an 
LGA-2011 board so I don't know what you might expect.

I'd stick with the 290 over the newer Radeons as the reset mechanisms 
are pretty hacky and custom to the card and I'm not sure what support 
there is for beyond the 290(X).

Finally, I am using a Intel 750 and a SuperMicro AOC-SLG3-2E4. That is 
an NVME drive and PLX U.2 switch/connector is it's working fine with 
ZFS. I'd go with that over any SATA based controller/drive. The 
interrupt overhead is much lower. It does take a slot (and IRQ) though. 
My board with a PLX switch helps, leaving just the IRQ whack-a-mole 
headache.

On 14 Sep 2015, at 8:46, Jouner Journer wrote:

> Hi Everyone,
>
> I'm looking to build a box which can run ZFS on Linux in hypervisor, 
> misc
> linux VMs, as well as a Windows vm for gaming.
>
> Is anyone doing video passthrough as well as running ZFSoL?  I've read 
> ZFS
> requires specific kernel versions in Arch and am wondering if I'll run 
> into
> compatibility issues.
>
> Also, this is the hardware I've selected... are there any blatant 
> problems
> that stick out?  I've been toying with the idea of different processor
> (less cores, more speed), but figured more cores is better...  how 
> about
> Radeon R9 390X... is it too new to be tested/confirmed?
>
> Given that it is xeon processor and motherboard supports VT-d/VT-x,
> motherboard and video card are uefi I'm hoping I will not run into too 
> many
> issues with passthrough... at least hardware limited issues ;)
>
> ASRock EPC612D8A-TB SSI ATX Server Motherboard Socket LGA 2011 R3
> Intel Xeon E5-2630 v3 Haswell-EP 2.4 GHz LGA 2011-3 85W 
> BX80644E52630V3
> Server Processor
> SAPPHIRE Radeon R9 390X 100381OCL 8GB 512-Bit GDDR5 PCI Express 3.0 
> 2.2
> Slot, ATX Tri-X OC Version (UEFI) Video Card
>
> SeaSonic Platinum-1000 1000W PSU
> Intel 535 Series SSDSC2BW180H601 2.5" 180GB SATA III MLC SSD
> Intel 535 Series SSDSCKJW360H601 M.2 360GB SATA III MLC SSD
> Kingston 64GB (4 x 16GB) 288-Pin DDR4 SDRAM ECC Registered DDR4 2133 
> (PC4
> 17000) Server Memory Model KVR21R15D4K4/64
>
>
> Thanks in advance!
> _______________________________________________
> vfio-users mailing list
> vfio-users at redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users