a comparison of ext3, jfs, and xfs on hardware raid

[Jeffrey W. Baker]

On Thu, 2005-07-14 at 12:33 -0600, Andreas Dilger wrote:
> On Jul 13, 2005  17:12 -0700, Jeffrey W. Baker wrote:
...
> > The system is using an Areca hardware raid controller with 5 7200RPM
> > SATA disks.  The RAID controller has 128MB of cache and the disks each
> > have 8MB.  The cache is write-back.  The system is Linux 2.6.12 on amd64
> > with 1GB system memory.
...
> > Next I used pgbench to test parallel random I/O.  pgbench has
> > configurable number of clients and transactions per client, and can
> > change the size of its database.  I used a database of 100 million
> > tuples (scale factor 1000).  I times 100,000 transactions on each
> > filesystem, with 10 and 100 clients per run.  Figures are in
> > transactions per second.
> > 
> >               ext3  jfs  xfs
> > 10 Clients      55   81   68
> > 100 Clients     61  100   64
> > 
> > Here XFS is not substantially faster but JFS continues to lead.  
> > 
> > JFS is roughly 60% faster than ext3 on pgbench and 40-70% faster on
> > bonnie++ linear I/O.
> 
> This is a bit surprising, I've never heard JFS as a leader in many
> performance tests.  Is pgbench at all related to dbench?  The problem
> with dbench is that for cases where the filesystem does no IO at all
> it reports a best result.  In real life the data has to make it to
> disk at some point.
...
> Try creating your ext3 filesystem with a larger journal, as Lustre does:
> 
> mkfs -J size=400 ...
> 
> size is in MB, 400 might be excessive for your setup - I'd be interested
> in hearing where the "sweet spot" is for journal size.  The latest e2fsprogs
> use 128MB as the largest default size (up from 32MB) for large filesystems.

The journal size doesn't seem to make any difference to pgbench, except
that 256MB seems to be the worst.  400MB and 32MB are roughly equal on
the pgbench workload.  400MB was the optimal journal size on the bonnie
++ workload.  

Perhaps it is silly to benchmark a database with its journal files on a
journalling filesystem, but here is the result.

  journal          pgbench tps         bonnie++ MB/s
--------------------------------------------------------
size |  mode   |  1  | 10 | 100 | write | rewrite | read
--------------------------------------------------------
32     journal                      57      35      112
32     ordered   28    51    57     83      33      101
32    writeback  34    70    88     57      31      103
64     journal                      55      33      113
64     ordered   29    52    61     84      33      100
64    writeback  32    69    87     59      31      100
128    journal                      52      33      109
128    ordered   32    54    62     86      34      102
128   writeback  34    70    88     61      32      102
256    journal                      54      30      110
256    ordered   28    51    60     90      34      106
256   writeback  29    64    79     59      31      104
400    journal                      52      28      108
400    ordered   26    49    59     89      33      104
400   writeback  32    70    87     60      32      101
---     ext2                105    118      32      107


-jwb