ext3 + fs > 2Tbyte

[Andreas Dilger]

On Nov 01, 2005  12:45 +1100, Vincent.McIntyre at csiro.au wrote:
> >It sounds like you have overflowed the end of the 2TB device limit and
> >clobbered the beginning of your filesystem.  This can happen if the
> >SCSI driver, kernel, or even ext3 isn't handling offsets > 2^31 properly.
> >I know RH has only recently started supporting ext3 filesystems > 2TB,
> >and it isn't clear that all drivers handle this properly yet.
> 
> This box is using the fusion mpt drivers as in 2.6.7 - mptbase,mptscsih
> etc. Do you recall any >2Tb issue being fixed in later kernels?

Sorry, I don't know, I've just heard of occasional problems in this area
and very few people reporting success.

> When the machine was last in a good state, the filesystem had 1.5Tbyte
> used, ie as far as I can tell nothing would have written past 2Tb,
> although I suppose there is no guarantee the space is used up in order
> of increasing offset.

No, it is "kind of" used in increasing offset, but not strictly so.

> >Please update your e2fsprogs to the latest.  You also need to use
> >"e2fsck -b 32768" (or multiple thereof) for such large filesystems.
> >I think newer e2fsprogs will print this message properly in that case.

You might also need to add "-B 4096".

> I downloaded 1.38 from sourceforge and built it. No change in behaviour.
> I tried e2fsck with block offsets from 1025 to 4194305 in steps of 1024.
> I also tried dumpe2fs with the same range of offsets, also nothing.
> 
> 
> Another question. The e2fsck(8) manpage says the superblocks are at -
>  Blocksize     -b
>  1k          8193
>  2k         16384
>  4k         32768
> Why is the superblock offset for 1k at 8193, not 8192?

Because the ext[23] superblock is at 1024 bytes offset from the
beginning of the device.  For 1kB blocksize this is a whole block
so the filesystem starts at block 1, while for larger blocksize
this is still in block 0.  Backup superblocks are at block offsets:

(blocksize * 8) * {3,5,7}^n, n={0,1,2,3...}

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.