Poor Performance WhenNumber of Files > 1M

Andreas Dilger adilger at clusterfs.com
Thu Aug 9 19:51:55 UTC 2007


Sean McCauliff <smccauliff at mail.arc.nasa.gov> wrote:
>I plan on having about 100M files totaling about 8.5TiBytes.   To see 
>how ext3 would perform with large numbers of files I've written a test 
>program which creates a configurable number of files into a configurable 
>number of directories, reads from those files, lists them and then 
>deletes them.  Even up to 1M files ext3 seems to perform well and scale 
>linearly; the time to execute the program on 1M files is about double 
>the time it takes it to execute on .5M files.  But past 1M files it 
>seems to have n^2 scalability.  Test details appear below.
>
>Looking at the various options for ext3 nothing jumps out as the obvious 
>one to use to improve performance.

Try increasing your journal size (mke2fs -J size=400), and having a lot
of RAM.

When you say "having about 100M files", does that mean "need to be
constantly accessing 100M files" or just "need to store a total of
100M files in this filesystem"?

The former means you need to keep the whole working set in RAM for
maximum performance, about 100M * (128 + 32) = 19GB of RAM.  The
latter is no problem, we have ext3 filesystems with > 250M files
in them.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.




More information about the Ext3-users mailing list