[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

ext2/ext3 directory handling

I have just been thru rebuilding a server that had been suffering 
severe performance problems.  I transferred the partitions to another 

hard drive, copying into freshly formatted filesystems.  This made 
a HUGE difference in performance.  One key directory structure 
stores email messages in an structure where there is a directory 
for each mail account, and the messages are stored in individual 
files.	Some of these had grown to contain thousands of messages. 
A couple of weeks ago we went thru a purge, and deleted message 
files for inactive accounts, and were able to dump 75% of the 
files.	Just doing that made a big performance improvement. Then 
a couple of days ago things got bad again, so I did the rebuild. 

My theory: these directories had obviously expanded to contain 
thousands of file entries.  From the fact that just copying to 
clean filesystems made such a difference, I assume that deleting 
the files probably does nothing to remove the vacant entries and 
shrink the directories themselves.  

All this  raises a few questions. . .

(1) Is the assumption that directories don't compress when deleting 
files correct?	How is this handled (in general terms)?

(2) Is there any difference between ext2 and ext3?  

(3) Does the htree code change the picture any (even 
though I don't use it, and won't until it is production) ?

(4) Is it possible that the directories themselves 
were fragmented?

(5) After doing a "mkdir" to create a new directory, how many 
file entries can it hold before it would be expanded to accept 
another file?  When a directory is expanded, how many additional 
file entries can be stored before needing another expansion?

(6) Say I have a directory containing some files, then I delete 
some files, and finally I start adding files.  Will new file 
entries use empty or vacated directory slots before expanding 
the directory?

(7) I am aware of e2defrag (latest version I have found is 0.73). 
Does this program (or any other any tool) perform any 
directory optimization that would affect this problem?

(8) If e2defrag would be helpful, has it/is it being brought 
forward to operate correctly with current (RH 8/9) systems?
I see some warnings about blocksise restrictions, etc.

(9) In designing new systems, are there some useful guidelines 
about the maximum number of files that can exist in a single 
directory without significant performance loss?  
I am interested in ext2, ext3, and htree.

A. Becker

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]