can ext3 directory entries be overwritten? -- Re: extremely slow "ls" on a cleared fatty ext3 directory on FC4/5

[Igor A. Valcov]

13.08.06 в 22:57 Robinson Tiemuqinke в письме писал(а):

> Hi, all,
>
>  I'm a newbie to ext3 file system, but what a pity if
> ext3 could not shrink after containing files and
> subdirectories get deleted.
>
>  If the ext3 directory could not shrink, then another
> natural question is: can the deleted directory entries
> be overwritten by new files/subdirs? The following is
> an example to detail my question:
>
> Suppose a directory named myDir hold 3 files a, b, c.
> then the ext3 file system will create 5 entries under
> myDir, ".", "..", 'a', 'b', 'c'.
>
> lets say I remove file a and b, then the entries on
> et3 will looks as:
>
>  '.', '..', '#a', '#b', 'c'. with '#?' means the file
> is delteded.
>
> at last I added three files d ,e,f, which entries
> should the ext3 file system create for myDir? case A,
> or case B? it looks like case A has better performance
> and is not hard to implement. But I'm not sure how
> ext3 picks.
>
> case A, '#a', '#b' entries are reused, so only one
> more entries are created.
>
> '.', '..', 'd', 'e', 'c', 'f'.
>
> case B, '#a', '#b' are not reused, instead, 3 more
> entries are created. so total entries are:
>
> '.', '..', '#a', '#b', 'c', 'd', 'e', 'f'.
>
> Please help, I'll be more than appreciated if any one
> can point out ext3 howtos and introduction links.
> Thanks a lot.
>
> --- John Wendel <jwendel10 at comcast.net> wrote:
>
>> Robinson Tiemuqinke wrote:
>> >  Hi,
>> >
>> >  A stupid flat directory /tmp holding 5 millon
>> files, the directory
>> >  locates on a ext3 file system with dir_index
>> feature turned on. The
>> >  running Linux are FC4 and FC5.
>> >
>> >  The files are just directly under /tmp, not in
>> any subdirectories --
>> >  they are results of mis-operations of users.
>> >
>> >  Then a 'ls' or 'find' command will take one hour
>> to finish, a lot of
>> >  other applications on the computer boxes are
>> affected.
>> >
>> >  I managed to have deleted the files one by one
>> with a 'find . |xargs
>> >  rm -rf' similar command in about 10 hours. but
>> after a file system
>> >  sync, it still take me 20 minutes to list the
>> cleaned /tmp directory
>> >  again -- even now the directory holds only 8
>> files total.
>> >
>> >  so I try to 'ls' the directory itself (not any
>> files and
>> >  subdirectories on it) and find that its size is
>> stupidly large (it is
>> >  131M even after deletion) compared with 4K for
>> normal directories.
>> >
>> >  -bash-3.00# ls -alFdh /tmp* drwxrwxrwt  4 root
>> staff 4.0K Aug 12
>> >  23:17 new_tmp/ drwxrwxrwt  4 root  staff 131M Aug
>> 12 20:30 tmp/
>> >
>> >  Anyone know why the former fatty directory still
>> looks unchanged and
>> >  takes hours to traverse even after 99.999999%
>> files got removed?
>> >
>> >  If there are any ways to fix this kind of problem
>> without rebooting
>> >  machine? I'm afraid of the commands "rsync -avHn
>> /tmp/ /new_tmp/; rm
>> >  -rf /tmp/ && mv /new_tmp/ /tmp" because other
>> applications are
>> >  accessing /tmp/ as well.
>> >
>> >  Please help. Thanks a lot.
>>
>> EXT3 directories grow but they don't shrink.
>> Rebooting won't fix this
>> problem.
>> The only fix that I know is to delete the old
>> directory and create a new
>> one.
>> BTW, XFS automatically shrinks directories (but has
>> its own set of
>> problems).

Use option -D in e2fsck for optimize directories.

        -D     Optimize  directories  in filesystem.  This option causes  
e2fsck to try to optimize
               all directories, either by reindexing them if  the   
filesystem  supports  directory
               indexing,   or  by  sorting and compressing directories for  
smaller directories, or
               for filesystems using traditional linear directories.

-- 
Igor A. Valcov