Poor Performance WhenNumber of Files > 1M

Sean McCauliff smccauliff at mail.arc.nasa.gov
Thu Aug 2 01:55:53 UTC 2007


Hi all,

I plan on having about 100M files totaling about 8.5TiBytes.   To see 
how ext3 would perform with large numbers of files I've written a test 
program which creates a configurable number of files into a configurable 
number of directories, reads from those files, lists them and then 
deletes them.  Even up to 1M files ext3 seems to perform well and scale 
linearly; the time to execute the program on 1M files is about double 
the time it takes it to execute on .5M files.  But past 1M files it 
seems to have n^2 scalability.  Test details appear below.

Looking at the various options for ext3 nothing jumps out as the obvious 
one to use to improve performance.

Any recommendations?

Thanks!
Sean

------
Parameter one is number of files, parameter two is number of directories 
to write into.

Dell MD3000 + LVM2. 2x7 10k rpm SAS disks 128k stripe RAID-0 for 3.8 
TiBytes of total storage.  Fedora Core 6 x86_64.  2xQuad Core Xeon. 
Default mount and ext3 options used.

[root at galaxy filestore]# time /soc/abyss/test/fileSystemTest.pl 10000 1000

real    0m1.054s
user    0m0.128s
sys     0m0.382s

[root at galaxy filestore]# time /soc/abyss/test/fileSystemTest.pl 1000000 1000

real    1m0.938s
user    0m12.203s
sys     0m40.358s
[root at galaxy filestore]# time /soc/abyss/test/fileSystemTest.pl 10000000 
1000

real    13m39.881s
user    2m6.645s
sys     7m26.665s
[root at galaxy filestore]# time /soc/abyss/test/fileSystemTest.pl 20000000 
1000

real    44m46.359s
user    4m22.911s
sys     17m2.792s





More information about the Ext3-users mailing list