[linux-lvm] LVM Performance effects?

[Bret Mckee]

I have set up a file server with LVM on top of RAID 5, and seem to be
having LVM related performance issue.

Interesting details:
-Processor:  Intel(R) Pentium(R) 4 CPU 2.80GHz with hyperthreading
-disks - 4 X Seagate 300 GB
  - 2 are SATA, connected to MB SATA connectors
  - 2 are PATA, connected to Promise add in card
- OS is centos 4.2
- uname -a -> Linux localhost.localdomain 2.6.9-22.ELsmp #1 SMP Sat
Oct 8 19:11:43 CDT 2005 i686 i686 i386 GNU/Linux
- all perf numbers obtained using time dd, and were ran 3 times.  Run
to run variance was within 1 second/run
- raid chunk size is 256k, file system block size is 4k, mke2fs was
done with stripe=64

I ran three tests, one to the raid device, one to the lvm device, one
through the file system.

raid device: 118.2 MB/s
lvm device: 49.43 MB/s
file system: 40.83 MB/s

I don't understand the LVM performance effect - I thougt LVM was a
pretty straight forward block remapper with negligible performance
impact. I'm stunned by the 58% performance penalty.

Any thoughts?  Any more data that would be helpful?

Thanks in advance,

Bret
---------------------

[root at localhost test]# time dd if=/dev/md1 of=/dev/null bs=1M
count=4096       
4096+0 records in
4096+0 records out

real    0m34.634s
user    0m0.021s
sys     0m13.467s


[root at localhost test]# time dd if=/dev/raid/mirror2 of=/dev/null bs=1M
count=4096
4096+0 records in
4096+0 records out

real    1m22.863s
user    0m0.021s
sys     0m14.384s

[root at localhost test]# time dd if=testfile of=/dev/null bs=1M
count=4096       
4096+0 records in
4096+0 records out

real    1m40.310s
user    0m0.020s
sys     0m9.996s