How To Recover From Creating >2TB ext3 Filesystem on MSDOS Partition Table?

[Jon Forrest]

(I've already sent this message to Ted Ts'o directly. I should
have sent it to this list first but I didn't know about it
until today. My apologies to Ted.)

Last Friday a system that I just inherited refused to mount
a file system that had been working fine for about 6 months.
This is on a Scientific Linux 4.3 system using a 2.6.9
kernel. This is another Linux distribution based on RHEL 4.
I don't think the actual hardware is relevant
here so I won't mention it. If there's more information you'd
like to see I'd be happy to provide it.

It turns out that this 4.2TB file system was created in an
msdos partition table, as shown below:

----
GNU Parted 1.6.19
Using /dev/sdb
(parted) p
Disk geometry for /dev/sdb: 0.000-4291443.000 megabytes
Disk label type: msdos
Minor    Start       End     Type      Filesystem  Flags
1          0.031  97137.567  primary   ext3
----

Running fsck fails as shown below:

----
e2fsck 1.35 (28-Feb-2004)
The filesystem size (according to the superblock) is 1098609033 blocks
The physical size of the device is 24867209 blocks
Either the superblock or the partition table is likely to be corrupt!
Abort<y>? yes

Error reading block 24870914 (Invalid argument) while doing inode scan.
----

I have 2 questions:

1) How did this system run just file for ~6 months using this
file system as a /home? I'm suspecting that the problem
actually occurred long ago when the file system allocated
meta or user data in blocks that are somehow unreachable
by fsck but exactly how this could have happened isn't
clear. Although it's too late now, I'd really like
to know what happened.

2) Given that this happened, how can I recover as many
files as possible from this file system? The professor
who owns this system had put his faith in hardware
RAID so he had never backed it up. He's very nervous
right now.

Any information or help you can provide would be
very much appreciated.

Cordially,
Jon Forrest
Unix Computing Support
College of Chemistry
Univ. of Cal. Berkeley
173 Tan Hall
Berkeley, CA
94720-1460
510-643-1032
jlforrest at berkeley.edu