resize2fs on LVM on MD raid on Fedora Core 3 - inode table conflicts in fsck

Sat Dec 11 07:57:08 UTC 2004

On Dec 09, 2004  11:55 -0700, Stephen Warren wrote:
> On the filesystem that I ran resize2fs on, is it now broken beyond 
> repair? I originally found the problem on a 'real' data volume. After 
> the resize, I can certainly mount it (read-only - didn't try rw) and 
> pull the data off it, but I assume that the fsck errors are legitimate 
> and the fs is corrupt, such that I should run mke2fs again from scratch?

You could either do a full e2fsck on it (it should be able to recover this
since the metadata that the inode table is overlapping is unused and e2fsck
will make a copy of the duplicate blocks) or you could use resize2fs to
shrink it back to the original size (that should also make the problem go
away).  Best to do this before doing any writes to the filesystem, and I
would suggest making a backup, but since this is already a backup...

> Well, I do have ext2online installed, although there's no e2fsadm. I'll 
> have to check for other RPMs...

It would be part of the lvm 1.0 tools if it was anywhere.  It is just a
convenience wrapper around lvextend and ext2online (or resize2fs for
unmounted resizes).

> So, I went ahead and tried ext2online, but I get a bunch of errors 
> during the execution - it's indicating my kernel doesn't have online 
> resize support compiled in. I would have assumed the FC(3) kernel did, 
> but I guess I should go check. Still, the debug trace from ext2online 
> indicates some other errors about allocating things before the kernel 
> problem.
> 
> Short version of log of ext2online:
> 
> ===============================
> ext2online: error reserving block 0x18813e
> finding 0x18813f in inode
> adding 0x18813f to inode
> ext2online: error reserving block 0x18813f
> finding 0x188140 in inode
> adding 0x188140 to inode
> ext2online: error reserving block 0x188140
> mark 16384 unavailable end inodes used
> ...
> ...
> Calling mount() for /dev/mapper/severn_vg0-test(/mnt/test) with 
> resize=2621440:319
> ext2online: resize failed while in kernel
> ext2online: Invalid argument
> ext2online: does the kernel support online resizing?

Having the kernel messages would be very useful here.  When I wrote
that patch it was an optional feature (under the ext3 filesystem option
in the config, maybe marked experimental), but looking at the current
Linus kernel it appears to always be enabled (i.e. no config option).

> SEVERN:/mnt# lvcreate -L 5G -n test severn_vg0
>   Logical volume "test" created
> 
> SEVERN:/mnt# mke2fs /dev/severn_vg0/test
> mke2fs 1.35 (28-Feb-2004)
> max_blocks 1342177280, rsv_groups = 40960, rsv_gdb = 319
> Filesystem label=
> OS type: Linux
> Block size=4096 (log=2)
> Fragment size=4096 (log=2)
> 655360 inodes, 1310720 blocks
> 65536 blocks (5.00%) reserved for the super user
> First data block=0
> Maximum filesystem blocks=1342177280
> 40 block groups
> 32768 blocks per group, 32768 fragments per group
> 16384 inodes per group
> Superblock backups stored on blocks:
>         32768, 98304, 163840, 229376, 294912, 819200, 884736
> 
> Writing inode tables: done
> inode.i_blocks = 20424, i_size = 4243456

Hmm, those are a bit strange, not sure where they are coming from.

> SEVERN:/mnt# ext2online -d -v /dev/severn_vg0/test
> ext2online v1.1.18 - 2001/03/18 for EXT2FS 0.5b
> ext2_open
> ext2_bcache_init
> new filesystem size 2621440
> ext2_determine_itoffset
> setting itoffset to +323
> group 2 inode table has offset 2, not 323
> group 4 inode table has offset 2, not 323
> group 6 inode table has offset 2, not 323
> group 8 inode table has offset 2, not 323
> group 10 inode table has offset 2, not 323
> ... same message through to group 39

These are normal - it just tells us that mke2fs didn't shift the inode
tables for non-backup groups.

> ext2_get_reserved
> Found 319 blocks in s_reserved_gdt_blocks
> using 319 reserved group descriptor blocks

This tells us mke2fs reserved 319 group descriptor blocks, so we can
resize up to 320 * 4096 / 32 = 40960 groups * 32MB/group = 1.2TB.  By
default it reserves enough blocks to resize 1024x the original fs size,
at the cost of 11MB of used space in the original 5GB filesystem.

> 40 old groups, 1 blocks
> 80 new groups, 1 blocks

We are staying within the original group descriptor block (makes the
resize a bit easier)

> ext2_ioctl: EXTEND group to 1310720 blocks
> ext2online: ext2_ioctl: Inappropriate ioctl for device

This is where it starts to look like online resize isn't available.
For compatibility with older kernels (2.4.x doing ext2 online resizing,
needs a patch) it doesn't just bail at this point since the resizing
mechanism from user->kernel space was different.

> creating group  49 with 32768 blocks (rsvd = 319, newgd =  1)
> using itoffset of 323
> new block bitmap is at 0x188141
> new inode bitmap is at 0x188142
> new inode table is at 0x188143-0x188342
> new group has 31933 free blocks
> new group has 16384 free inodes (512 blocks)
> mark superblock 0x188000 used
> mark group desc. 0x188001-0x188001 used
> mark reserved descriptors 0x188002-0x188140 used
> finding 0x188002 in inode
> adding 0x188002 to inode
> add 0 as direct block
> finding 0x188003 in inode
> adding 0x188003 to inode
> add 1 as direct block
> finding 0x188004 in inode
> adding 0x188004 to inode
> add 2 as direct block
> 
> ... same message for many inodes
> 
> ext2online: error reserving block 0x18813e
> finding 0x18813f in inode
> adding 0x18813f to inode
> ext2online: error reserving block 0x18813f
> finding 0x188140 in inode
> adding 0x188140 to inode
> ext2online: error reserving block 0x188140

This is probably some bad interaction between the ext2 resize mechanism
and the ext3 one.  I think at this point this ext2 online resize code
should be removed from ext2online, since it isn't really being supported
and is just causing grief to the world.

> ext2online: resizing to 2621440 blocks
>    ...flushing buffer 81/block 2588673
> Calling mount() for /dev/mapper/severn_vg0-test(/mnt/test) with 
> resize=2621440:319
> ext2online: resize failed while in kernel
> ext2online: Invalid argument
> ext2online: does the kernel support online resizing?

Here we probably get another kernel message like 'Unrecognized mount
option "resize" or missing value'.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://members.shaw.ca/adilger/             http://members.shaw.ca/golinux/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/ext3-users/attachments/20041211/1347edf2/attachment.sig>