ext3 with maildir++ = huge disk latency and high load

[Ted Ts'o]

On Thu, Sep 29, 2011 at 11:29:42AM +0400, Andrey wrote:
> Ok. Here are bonnie results on fresh installed Debian with 200GB
> FREE ext3 /home partitition (4 disks in RAID5 on HP Proliant DL380
> G4 server):
> 
> Version  1.96  ------Sequential Output------ --Sequential Input- --Random-
> Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr-
> --Block-- --Seeks--
> Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec
> %CP /sec %CP
> debian           2G   243  97 22555  10  8794   2  1810  97 120444
> 11 317.0   5
> Latency               135ms     967ms     723ms   26526us   13143us
> 586ms
> 
> Latency is also very bad according results. What is the reason?
> Hardware or ext3 itseld? Will try with xfs an ext4 and compare then.

Ext4 will do better than ext3; but I can predict to you up front that
xfs will do better than ext4, because it is better optimized for RAID
arrays at the moment.  Ext4 has superblock fields to store RAID 5
parameters, but the code to fully take advantage of those RAID
parameters is not fully implemented.

You should also take a look at your RAID parameters.  If your RAID
stripe size is too large, it will impact workloads such as mail
servers which typically involve small writes.  Have you considered
using RAID 10 over RAID 5?  It's not as efficient from a space
perspective but if you are primarily concerned about throughput and
latency, it's the way to go.

						- Ted