16TB ext3 mainstream - when?

[Andreas Dilger]

On Oct 04, 2006  16:40 +0100, Christian wrote:
> I don't get it: I thought ext2/3 filesystems (volumes) can be 32TiB in 
> size? At least that's what [0] says. If this is wrong, someone should 
> correct this information. Although I must admit that 16 TiB per fs makes 
> more sense, given that with a max blocksize of 4K and a max of 2^32 
> blocks we have 16TiB of data.... Where does this 2^32 limitation come 
> from anyway?

The 2^32 limit is a 32-bit integer number of blocks.  In older kernels
(i.e. anything except the latest -mm) there is a signed-int problem,
so the effective limit is 2^31 blocks.  With 1kB blocks this limit is
2TB (2^41 bytes), with 4kB blocks (most common) it is 8TB (2^43 bytes)
with 64kB blocks (PPC64, ia64, other large PAGE_SIZE systems) this limit
is 32TB (2^45 bytes).

The ext4 filesystem allows up to 2^48 blocks in the filesystem so the
limit is 2^60 bytes for 4kB blocks, and 2^64 bytes for 64kB blocks.

The major problem at this point is e2fsck time, which is about 1h/TB for
fast disks, at minimum (i.e. no major corruption found).  One of the
goals for future ext4 development is to include checksums into the fs
to allow online sanity checking, and also speed up e2fsck in various ways.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.