ext3 efficiency, larger vs smaller file system, lots of inodes...
jarmstrong at postpath.com
Tue May 19 17:08:37 UTC 2009
> -----Original Message-----
> From: Ric Wheeler [mailto:rwheeler at redhat.com]
> Sent: Tuesday, May 19, 2009 9:54 AM
> To: Joe Armstrong
> Cc: ext3-users at redhat.com
> Subject: Re: ext3 efficiency, larger vs smaller file system, lots of
> On 05/19/2009 12:28 PM, Joe Armstrong wrote:
> > -----Original Message-----
> > From: Eric Sandeen [mailto:sandeen at redhat.com]
> > Sent: Tuesday, May 19, 2009 9:21 AM
> > To: Joe Armstrong
> > Cc: ext3-users at redhat.com
> > Subject: Re: ext3 efficiency, larger vs smaller file system, lots of
> > Joe Armstrong wrote:
> >> (... to Nabble Ext3:Users - reposted by me after I joined the
> >> ext3-users mailing list - sorry for the dup...)
> >> A bit of a rambling subject there but I am trying to figure out if
> >> is more efficient at runtime to have few very large file systems (8
> >> TB) vs a larger number of smaller file systems. The file systems
> >> will hold many small files.
> >> My preference is to have a larger number of smaller file systems for
> >> faster recovery and less impact if a problem does occur, but I was
> >> wondering if anybody had information from a runtime performance
> >> perspective - is there a difference between few large and many
> >> file systems ? Is memory consumption higher for the inode tables if
> >> there are more small ones vs one really large one ?
> > It's the vfs that caches dentries& inodes; whether they come from
> > multiple filesystems or one should not change matters significantly.
> > The other downside to multiple smaller filesystems is space
> > when you wind up with half of them full and half of them empty, it
> > be hard to rearrange.
> > But the extra granularity for better availability and fsck/recovery
> > may be well worth it. It probably depends on what your application
> > doing and how it can manage the space. You might want to test
> > an 8T filesystem and see for yourself how long fsck will take...
> > be a while. Perhaps a very long while. :)
> >> Also, does anybody have a reasonable formula for calculating memory
> >> requirements of a given file system ?
> > Probably the largest memory footprint will be the cached dentries&
> > inodes, though this is a "soft" requirement since it's mostly just
> > Each journal probably has a bit of memory requirement overhead, but I
> > doubt it'll be a significant factor in your decision unless every
> > is at a premium...
> > -Eric
> How you do this also depends on the type of storage you use. If you
> multiple file systems on one physical disk (say 2 1TB partitions on a
> 2TB S-ATA
> disk), you need to be careful not to bash on both file systems at once
> since you
> will thrash the disk heads.
> In general, it is less of an issue with arrays, but still can have a
Just for completeness, we will be using Striped LUN's (RAID-6 underneath), so I hope that the striping will distribute the IO's while the RAID-6 device will provide the HA/recovery capabilities.
More information about the Ext3-users