[linux-lvm] Re: LVM and Raid5

[Mike Snitzer]

On Mon, Sep 21 2009 at 12:30pm -0400,
Jon Hardcastle <jd_hardcastle at yahoo.com> wrote:

> --- On Mon, 21/9/09, Mike Snitzer <snitzer at gmail.com> wrote:
> 
> > From: Mike Snitzer <snitzer at gmail.com>
> > Subject: Re: LVM and Raid5
> > To: "Michal Soltys" <soltys at ziu.info>
> > Cc: "Linux Raid Study" <linuxraid.study at gmail.com>, linux-raid at vger.kernel.org, linux-lvm at redhat.com
> > Date: Monday, 21 September, 2009, 3:33 PM
> > On Thu, Sep 17, 2009 at 8:37 AM,
> > Michal Soltys <soltys at ziu.info>
> > wrote:
> > > Linux Raid Study wrote:
> > >>
> > >> Hello:
> > >>
> > >> Has someone experimented with LVM and Raid5
> > together (on say, 2.6.27)?
> > >> Is there any performance drop if LVM/Raid5 are
> > combined vs Raid5 alone?
> > >>
> > >> Thanks for your inputs!
> > >
> > > Few things to consider when setting up LVM on MD
> > raid:
> > >
> > > - readahead set on lvm device
> > >
> > > It defaults to 256 on any LVM device, while MD will
> > set it accordingly to
> > > the amount of disks present in the raid. If you do
> > tests on a filesystem,
> > > you may see significant differences due to that. YMMV
> > depending on the type
> > > of used benchmark(s).
> > >
> > > - filesystem awareness of underlying raid
> > >
> > > For example, xfs created on top of raid, will
> > generally get the parameters
> > > right (stripe unit, stripe width), but if it's xfs on
> > lvm on raid, then it
> > > won't - you will have to provide them manually.
> > >
> > > - alignment between LVM chunks and MD chunks
> > >
> > > Make sure that extent area used for actual logical
> > volumes start at the
> > > boundary of stripe unit - you can adjust the LVM's
> > metadata size during
> > > pvcreate (by default it's 192KiB, so with non-default
> > stripe unit it may
> > > cause issues, although I vaguely recall posts that
> > current LVM is MD aware
> > > during initialization). Of course LVM must itself
> > start at the boundary for
> > > that to make any sense (and it doesn't have to be the
> > case - for example if
> > > you use partitionable MD).
> > 
> > All of the above have been resolved in recent LVM2
> > userspace (2.02.51
> > being the most recent release with all these
> > addressed).  The last
> > issue you mention (partitionable MD alignment offset) is
> > also resolved
> > when a recent LVM2 is coupled with Linux 2.6.31 (which
> > provides IO
> > Topology support).
> > 
> > Mike
> > --
> 
> When you say 'resolved' are we talking automatically? if so, when the
> volumes are created... etc etc?

Yes, automatically when the volumes are created.

The relevant lvm.conf options (enabled by default) are:

devices/md_chunk_alignment (useful for LVM on MD w/ Linux < 2.6.31)
devices/data_alignment_detection
devices/data_alignment_offset_detection

readahead defaults to "auto" in lvm.conf:
activation/readahead