[Fedora] Re: Alternatives to du

Mark Haney mhaney at ercbroadband.org
Tue Apr 10 16:39:29 UTC 2007


Alan Cox wrote:
>> this is exactly my problem I believe.  I have a 4TB RAID SAN and the 
>> problem directory holds well over a million small files.  Even a 
>> directory under that directory, du takes almost 40 minutes to total up 74GB.
> 
> The directory code in ext2 and ext3 isn't designed for a million files in
> a single directory. Its a bizzare corner case. If you split the files
> into subdirectories (eg by a hash) you'll get far saner numbers
> 

Unfortunately, in this case, the directory structure is what it is.  I 
can't change it without a massive amount of pain.

However, on further review, my '6 hour' time has dropped significantly 
over the course of the last month or so.  I believe this is due to a 
re-organizing when data is accessed on that array.  Not to mention I did 
have a failed drive in there.  I don't think that had any bearing on it 
since I had a hot spare that took up the load when that drive went bad. 
  Granted the rebuild time was long, but I know the stats I've pulled 
recently have been better.  Even now though, it still takes 37 minutes 
to du a directory of 74GB. Maybe I"m just missing something.


-- 
Ita erat quando hic adveni.

Mark Haney
Sr. Systems Administrator
ERC Broadband
(828) 350-2415




More information about the fedora-list mailing list