[linux-lvm] Best way to run LVM over multiple SW RAIDs?

[Gionatan Danti]

Il 08-12-2019 00:14 Stuart D. Gathman ha scritto:
> On Sat, 7 Dec 2019, John Stoffel wrote:
> 
>> The biggest harm to performance here is really the RAID5, and if you
>> can instead move to RAID 10 (mirror then stripe across mirrors) then
>> you should be a performance boost.
> 
> Yeah, That's what I do.  RAID10, and use LVM to join together as JBOD.
> I forgot about the raid 5 bottleneck part, sorry.
> 
>> As Daniel says, he's got lots of disk load, but plenty of CPU, so the
>> single thread for RAID5 is a big bottleneck.
> 
>> I assume he wants to use LVM so he can create volume(s) larger than
>> individual RAID5 volumes, so in that case, I'd probably just build a
>> regular non-striped LVM VG holding all your RAID5 disks.  Hopefully
> 
> Wait, that's what I suggested!
> 
>> If you can, I'd get more SSDs and move to RAID1+0 (RAID10) instead,
>> though you do have the problem where a double disk failure could kill
>> your data if it happens to both halves of a mirror.
> 
> No worse than raid5.  In fact, better because the 2nd fault always
> kills the raid5, but only has a 33% or less chance of killing the
> raid10.  (And in either case, it is usually just specific sectors,
> not the entire drive, and other manual recovery techniques can come 
> into
> play.)

While I agree with both (especially regarding RAID10), I propose another 
setup: and MD RAID0 of the eight MD RAID5 arrays.
If I remember correctly, LVM striping code is based on device mapper 
rather than MD RAID code. Maybe the latter is more efficient at striping 
on fast NVMe drives?

Regards.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti at assyoma.it - info at assyoma.it
GPG public key ID: FF5F32A8