[long] major problems on fs; e2fsck running out of memory

[Keith Keller]

On Mon, Jun 02, 2014 at 07:30:25AM -0400, Theodore Ts'o wrote:
> Yeah, it would have been nice to have preserved the outputs from
> earlier e2fsck runs just so we could see what e2fsck did that
> apparently ended up overwriting parts of the block group descriptors.

I think I still have most of my e2fsck outputs.  The only ones I don't
have, which might be the most helpful, are the initial one or two, when
I thought the recovery would be easy.  I can post them somewhere if you
think it'd be helpful in tracking down a bug (they are probably too long
to post to the list).

> One of the things that might have detected the problem sooner, and
> perhaps allowed you to recover more smoothly, would be to try running
> e2fsck immediately after running resize2fs.  With the vintage kernel
> and e2fsprogs shipping with the version of CentOS you are apparently
> using, online resizing is probably safer than off-line --- although if
> you are using the 1.42.10 version of resize2fs and the 2.6.32 based
> kernel, I'd probably sugges off-line resizes as being more safe.  And
> either way, running e2fsck on the file system after the resize is
> probably a good idea.

Well, if you're going to run e2fsck, you may as well do an offline
resize, since you have to umount the filesystem anyway.

Just to clarify, the kernel I'm running is actually an OpenVZ kernel, so
I wouldn't rely totally on the version number.  I want to try to find
their changelogs to see what sort of bug fixes they've included in their
kernel.  The resize2fs was done under an older OpenVZ kernel; the
current kernel is the latest OpenVZ kernel.

Ah, here is the latest changelog:

https://openvz.org/Download/kernel/rhel6/042stab090.2

The kernel which was running for the resize was this one (which is
definitely old and crufty):

https://openvz.org/Download/kernel/rhel6/042stab055.10

> I'd try using the backup superblock approach, but if that doesn't
> work, yes, that's probably a reasonable conclusion.

Great!  Again, thank you so much for all your help.

--keith

-- 
kkeller at wombat.san-francisco.ca.us