[linux-lvm] poor read performance on rbd+LVM, LVM overload
dwm37 at cam.ac.uk
Thu Oct 17 09:06:55 UTC 2013
On 16/10/2013 17:16, Sage Weil wrote:
> I'm not sure what options LVM provides for aligning things to the
> underlying storage...
There is a generic kernel ABI for exposing performance properties of
block devices to higher layers, so that they can automatically tune
themselves according to those performance properties, and report their
performance properties to users higher up the stack.
LVM supports both reading this data from underlying physical devices,
configuring itself as appropriate --- as well as reporting this data to
users of LVs, so that they can, too.
(For example, mkfs.xfs uses libblkid to automatically select the optimal
stripe-size, stride width, etc. of an LVM volume sitting on top of an MD
A good starting point appears to be:
If Ceph RBD block devices don't currently expose this information, that
should be a relatively simple addition that will result in all higher
layers, whether LVM or a native filesystem, automatically tuning
themselves at creation-time for the RBD's performance characteristics.
(As an aside, it's possible that OSD journalling performance could also
be improved by teaching it to heed this topology information. I can
imagine that when writing directly to block devices it may be possible
to improve performance, such as when using LVM-on-an-SSD, or a DOS
partition on a 4k-sector SATA disk.)
~ ~ ~
In the mean time, the documentation I found for LVM2 suggests that the
`pvcreate` command supports the "--dataalignment" and
The former should be the RBD object size, e.g. 4MB by default. In this
case, you'll also need to set the latter compensate for the offset
introduced by the GPT place-holder partition table at the start of the
device so that LVM data extents begin on an object boundry.
David McBride <dwm37 at cam.ac.uk>
Unix Specialist, University Computing Service
More information about the linux-lvm