Is there a way to cache a file system listing?

Kevin Strong kstrong at criminalinfo.net
Wed Jun 21 20:43:44 UTC 2006


Theodore Tso wrote:

>On Wed, Jun 21, 2006 at 03:15:38PM -0400, Kevin Strong wrote:
>  
>
>>Justin Piszcz wrote:
>>    
>>
>>>I have a filesystem that has roughly ~600,000 files on it and every 
>>>time a new client rsyncs this tree (after 10-20 minutes when the cache 
>>>has expired) then it takes 5-10 minutes to re-traverse the tree during 
>>>a new rsync.  Is there a way, other than running find /path every 
>>>minute or so, to keep the listing in memory so the rsyncs would run 
>>>much faster?
>>>      
>>>
>>I second this.  I am having the same exact issue right now.  Any 
>>suggestions would be appreciated.
>>    
>>
>
>Are you sure the time is actually re-traversing the tree, and not
>calculating the per-file checksums?  Running an rsync server does
>present a big load on the server, but traditionally it's not been a
>matter of whether or not the directories have been cached, but rather
>the fact that rsync doesn't cache the per-file checksums, and has to
>recalculate them all each time a new client connects.
>
>						- Ted
>
>  
>

For me, this is on the client side, not the server side.  Sorry; should 
have clarified.  It's definitely an issue with the directory tree 
falling out of the FS cache.  Are you aware of a solution to this?

Kevin Strong
Criminal Information Network, Inc.




More information about the Ext3-users mailing list