a comparison of ext3, jfs, and xfs on hardware raid

Andreas Dilger adilger at clusterfs.com
Thu Jul 14 18:33:30 UTC 2005

On Jul 13, 2005  17:12 -0700, Jeffrey W. Baker wrote:
> I'm setting up a new file server and I just can't seem to get the
> expected performance from ext3.  Unfortunately I'm stuck with ext3 due
> to my use of Lustre.  So I'm hoping you dear readers will send me some
> tips for increasing ext3 performance.
> The system is using an Areca hardware raid controller with 5 7200RPM
> SATA disks.  The RAID controller has 128MB of cache and the disks each
> have 8MB.  The cache is write-back.  The system is Linux 2.6.12 on amd64
> with 1GB system memory.
> Using bonnie++ with a 10GB fileset, in MB/s:
>          ext3    jfs    xfs
> Read     112     188    141
> Write     97     157    167
> Rewrite   51      71     60
> These number were obtained using the mkfs defaults for all filesystems
> and the deadline scheduler.  As you can see JFS is kicking butt on this
> test.

One thing that is important for Lustre is performance of EAs.  See
http://samba.org/~tridge/xattr_results/ for a comparison.  Lustre
uses large inodes (-I 256 or larger) to store the EAs efficiently.

> Next I used pgbench to test parallel random I/O.  pgbench has
> configurable number of clients and transactions per client, and can
> change the size of its database.  I used a database of 100 million
> tuples (scale factor 1000).  I times 100,000 transactions on each
> filesystem, with 10 and 100 clients per run.  Figures are in
> transactions per second.
>               ext3  jfs  xfs
> 10 Clients      55   81   68
> 100 Clients     61  100   64
> Here XFS is not substantially faster but JFS continues to lead.  
> JFS is roughly 60% faster than ext3 on pgbench and 40-70% faster on
> bonnie++ linear I/O.

This is a bit surprising, I've never heard JFS as a leader in many
performance tests.  Is pgbench at all related to dbench?  The problem
with dbench is that for cases where the filesystem does no IO at all
it reports a best result.  In real life the data has to make it to
disk at some point.

See http://sudhaa.com/~benchmark/ext3/newtiobenchresults.ext3gold/newtiobench/newtiobench.html
for a comparison of ext3, xfs, jfs in the mode that Lustre runs in
(specifically column 7, 14, 18).

> Are there any tunables that I might want to adjust to get better
> performance from ext3?

Try creating your ext3 filesystem with a larger journal, as Lustre does:

mkfs -J size=400 ...

size is in MB, 400 might be excessive for your setup - I'd be interested
in hearing where the "sweet spot" is for journal size.  The latest e2fsprogs
use 128MB as the largest default size (up from 32MB) for large filesystems.

Cheers, Andreas
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

More information about the Ext3-users mailing list