[vfio-users] VFIO + ZFS build

Tue Sep 15 07:22:15 UTC 2015

I once put the ZFS workload on a dedicated Debian VM that I passthrough my IBM M1015 into. Worked very well although I had this message about the controller’s BIOS at first and then I had to pass the rom file on the QEMU command line. The Debian VM could see all my disks just as on my native (opening Disk Utility resulting the same). I wouldn’t recommend passing through the onboard SATA controller to a VM, unless the motherboard has additional SATA controller like Marvell or ASmedia such that I can put the Hypervisor OS on the Intel/AMD SATA controller and Marvel/ASmedia SATA controller passed through to a VM.

-- 
Okky Hendriansyah

On September 15, 2015 at 14:01:29, francesco dicarlo (evilsephiroth at gmail.com) wrote:

Honestly, I would go the other way. Passing sata can be a nightmare while pci passthrough for non-vga card has always worked for me. Currently passing a dvb-S pci express card  (DVBSky S952 dual dvb-s tuner) to an ubuntu vm with VDR (Video Disk Recorder) without problem.

2015-09-15 6:46 GMT+02:00 Jouner Journer <journerforums at gmail.com>:
Thanks for the info.
Are you running zfs in the hypervisor or in a vm?
Using zfs for root partition?

I was planning on using onboard sata for zfs.  I suppose passing that controller to a vm would be problematic

On Tuesday, September 15, 2015, <globalgorrilla at fastmail.fm> wrote:
Just for reference it's a Quadro K2200 (not a Quadro 2000) I have. I've also used both a R9 290 and R9 260X equally well. Nothing newer than that.

On 14 Sep 2015, at 12:58, globalgorrilla at fastmail.fm wrote:

I'm running ZFSoL on 4.1.6 and using an R9 290 and Quadro 2000.

Both cards run well (both reset and are fully supported in the VM with HyperV extensions).

ZFS is memory hungry and I could use more than 32 GB RAM.

ZFSoL as of a couple days supports Linux 4.2 but my QEMU/OVMF VMs don't work with 4.2. I've heard some here say it might be OVMF.

My biggest issue once I figured out the quirks for my MB has been that of IRQ sharing on my Z97 board (Z97-WS).

The Quadro especially does not like to share an IRQ (with any other device). The Radeons seem less drastically affected but some combinations of Radeons don't work with shared IRQs. That seems to be something with the cards and not specific to vfio or Linux. ASUS documents their IRQ combinations but in reality once you disable some onboard devices it's a crapshoot as to what IRQs get assigned. Your board's manual doesn't mention IRQ assignments and I don't have an LGA-2011 board so I don't know what you might expect.

I'd stick with the 290 over the newer Radeons as the reset mechanisms are pretty hacky and custom to the card and I'm not sure what support there is for beyond the 290(X).

Finally, I am using a Intel 750 and a SuperMicro AOC-SLG3-2E4. That is an NVME drive and PLX U.2 switch/connector is it's working fine with ZFS. I'd go with that over any SATA based controller/drive. The interrupt overhead is much lower. It does take a slot (and IRQ) though. My board with a PLX switch helps, leaving just the IRQ whack-a-mole headache.

On 14 Sep 2015, at 8:46, Jouner Journer wrote:

Hi Everyone,

I'm looking to build a box which can run ZFS on Linux in hypervisor, misc
linux VMs, as well as a Windows vm for gaming.

Is anyone doing video passthrough as well as running ZFSoL?  I've read ZFS
requires specific kernel versions in Arch and am wondering if I'll run into
compatibility issues.

Also, this is the hardware I've selected... are there any blatant problems
that stick out?  I've been toying with the idea of different processor
(less cores, more speed), but figured more cores is better...  how about
Radeon R9 390X... is it too new to be tested/confirmed?

Given that it is xeon processor and motherboard supports VT-d/VT-x,
motherboard and video card are uefi I'm hoping I will not run into too many
issues with passthrough... at least hardware limited issues ;)

ASRock EPC612D8A-TB SSI ATX Server Motherboard Socket LGA 2011 R3
Intel Xeon E5-2630 v3 Haswell-EP 2.4 GHz LGA 2011-3 85W BX80644E52630V3
Server Processor
SAPPHIRE Radeon R9 390X 100381OCL 8GB 512-Bit GDDR5 PCI Express 3.0 2.2
Slot, ATX Tri-X OC Version (UEFI) Video Card

SeaSonic Platinum-1000 1000W PSU
Intel 535 Series SSDSC2BW180H601 2.5" 180GB SATA III MLC SSD
Intel 535 Series SSDSCKJW360H601 M.2 360GB SATA III MLC SSD
Kingston 64GB (4 x 16GB) 288-Pin DDR4 SDRAM ECC Registered DDR4 2133 (PC4
17000) Server Memory Model KVR21R15D4K4/64

Thanks in advance!
_______________________________________________
vfio-users mailing list
vfio-users at redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

_______________________________________________
vfio-users mailing list
vfio-users at redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

_______________________________________________
vfio-users mailing list
vfio-users at redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

_______________________________________________  
vfio-users mailing list  
vfio-users at redhat.com  
https://www.redhat.com/mailman/listinfo/vfio-users  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20150915/2e2f3b1a/attachment.htm>