Poor Performance WhenNumber of Files > 1M

Sean McCauliff smccauliff at mail.arc.nasa.gov
Thu Aug 9 20:04:03 UTC 2007


> Try increasing your journal size (mke2fs -J size=400), and having a lot
> of RAM.
> 
> When you say "having about 100M files", does that mean "need to be
> constantly accessing 100M files" or just "need to store a total of
> 100M files in this filesystem"?
Likely only 10M will be accessed at any time.

> 
> The former means you need to keep the whole working set in RAM for
> maximum performance, about 100M * (128 + 32) = 19GB of RAM.  The
> latter is no problem, we have ext3 filesystems with > 250M files
> in them.
The system has 16G of RAM; getting 32G in the future is a possibility. 
  Where do you get 128 + 32 from?  Is 128 the inode size?  this is 
running a 64bit os.  Does that change the memory requirements?

Thanks, I will try your suggestion.

Sean





More information about the Ext3-users mailing list