[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Poor Performance WhenNumber of Files > 1M

dumpe2fs reports that dir_index option is enabled. But thank you for the suggestion.


Bruno Wolff III wrote:
On Wed, Aug 01, 2007 at 18:55:53 -0700,
  Sean McCauliff <smccauliff mail arc nasa gov> wrote:
Hi all,

I plan on having about 100M files totaling about 8.5TiBytes. To see how ext3 would perform with large numbers of files I've written a test program which creates a configurable number of files into a configurable number of directories, reads from those files, lists them and then deletes them. Even up to 1M files ext3 seems to perform well and scale linearly; the time to execute the program on 1M files is about double the time it takes it to execute on .5M files. But past 1M files it seems to have n^2 scalability. Test details appear below.

Looking at the various options for ext3 nothing jumps out as the obvious one to use to improve performance.

Any recommendations?

Did you make sure directory indexing is available? I think that is the
default now for ext3, but maybe it wasn't turned on for your test.

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]