Ext3 and 3ware RAID5

[Joshua Baker-LePain]

I run a decent amount of 3ware hardware, all under centos-4.  There seems 
to be some sort of fundamental disagreement between ext3 and 3ware's 
hardware RAID5 mode that trashes write performance.  As a representative 
example, one current setup is 2 9550SX-12 boards in hardware RAID5 mode 
(256KB stripe size) with a software RAID0 stripe on top (also 256KB 
chunks).  bonnie++ results look like this:

mount -t ext3
175 MB/s writes, 352 MB/s reads

mount -t ext3 -o data=writeback
185 MB/s writes, 254 MB/s reads

mount -t ext2
340 MB writes, 266 MB/s reads

XFS on this hardware gets (untuned) about 300 MB/s writes and 400 MB/s 
reads.  The hardware itself is capable of more, and those results are 
representative of several different configs of hardware and software RAID 
options.

Any ideas as to what leads to ext3's performance hit?  I've tested *lots* 
of configurations.  It's not the md layer -- 1 card setups see the same 
performance hit.

Thanks.

-- 
Joshua Baker-LePain
Department of Biomedical Engineering
Duke University