ext3 efficiency, larger vs smaller file system, lots of inodes...

[Eric Sandeen]

Joe Armstrong wrote:
> (... to Nabble Ext3:Users - reposted by me after I joined the
> ext3-users mailing list - sorry for the dup...)
> 
> A bit of a rambling subject there but I am trying to figure out if it
> is more efficient at runtime to have few very large file systems (8
> TB) vs a larger number of smaller file systems.  The file systems
> will hold many small files.
> 
> My preference is to have a larger number of smaller file systems for
> faster recovery and less impact if a problem does occur, but I was
> wondering if anybody had information from a runtime performance
> perspective  - is there a difference between few large and many small
> file systems ?  Is memory consumption higher for the inode tables if
> there are more small ones vs one really large one ?

It's the vfs that caches dentries & inodes; whether they come from
multiple filesystems or one should not change matters significantly.

The other downside to multiple smaller filesystems is space management,
when you wind up with half of them full and half of them empty, it may
be hard to rearrange.

But the extra granularity for better availability and fsck/recovery time
may be well worth it.  It probably depends on what your application is
doing and how it can manage the space.  You might want to test filling
an 8T filesystem and see for yourself how long fsck will take... it'll
be a while.  Perhaps a very long while.  :)

> Also, does anybody have a reasonable formula for calculating memory
> requirements of a given file system ?

Probably the largest memory footprint will be the cached dentries &
inodes, though this is a "soft" requirement since it's mostly just cached.

Each journal probably has a bit of memory requirement overhead, but I
doubt it'll be a significant factor in your decision unless every byte
is at a premium...

-Eric

> Thanks. Joe