ext4 and unexpected eh_depth

Wed Feb 4 08:41:32 UTC 2009

Hi, I'm running Debian lenny with linux-image-2.6.26-1-amd64  (deb
2.6.26-11).  I have a lvm stripe over three sata disks (3.5TB total)
that is shared over NFS, and I getting following errors

EXT4-fs error (device dm-0): ext4_ext_search_right: bad header in inode #269200: unexpected eh_depth - magic f30a,entries 18, max 340(0), depth 1(2)

An user is having errors in concatenating large files (100+GB): basicly seems that the the resulting file is right size
and ends with right data, but anyway he gets following error: 
cat: write error: Input/output error
on system that has imported partition over NFS. I'm not sure if the file he was accessing did had the same inode.

And once I got BUG below.  I cannot right now upgrade system as there are some long-running analysis running,
but can do some tests at some point, and upgrade in few days.

------------[ cut here ]------------
kernel BUG at fs/jbd2/transaction.c:1161!
invalid opcode: 0000 [1] SMP 
CPU 0 
Modules linked in: nfsd lockd nfs_acl auth_rpcgss sunrpc exportfs ext4dev jbd2 crc16 dag(P) ipv6 dm_mod dagmem(P) loop snd_pcm snd_timer snd soundcore snd_page_alloc intel_rng i2c_i801 rng_core i2c_core parport_pc parport pcspkr iTCO_wdt container shpchp pci_hotplug i5000_edac button edac_core evdev ext3 jbd mbcache sd_mod ahci libata scsi_mod dock floppy ehci_hcd uhci_hcd e1000e thermal processor fan thermal_sys
Pid: 3501, comm: nfsd Tainted: P          2.6.26-1-amd64 #1
RIP: 0010:[<ffffffffa021e5f5>]  [<ffffffffa021e5f5>] :jbd2:jbd2_journal_dirty_metadata+0x5f/0xe3
RSP: 0018:ffff810009543c90  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff81007cd38880 RCX: 00000000ffffffc0
RDX: ffff81001f9e74c0 RSI: ffff81007cd38880 RDI: ffff8100425833a8
RBP: ffff810045b7c490 R08: ffff810034a5a4d8 R09: ffffffffa024be70
R10: 000000000000005c R11: ffff81007cd38880 R12: ffff81003790c000
R13: ffff8100425833a8 R14: 00000000000020dc R15: 0000000000000000
FS:  00007f13c048e6e0(0000) GS:ffffffff8053b000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f8404606210 CR3: 00000000049a0000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process nfsd (pid: 3501, threadinfo ffff810009542000, task ffff81007f05e850)
Stack:  0000000000000000 ffff81000948af80 ffff8100425833a8 ffff81007cd38880
 ffffffffa024be70 ffffffffa0242122 ffff810034a5a4d8 ffff81000948af80
 ffff810005a34b80 ffff810004ca4c90 ffff81007d5c8c00 ffffffffa0230dab
Call Trace:
 [<ffffffffa0242122>] ? :ext4dev:__ext4_journal_dirty_metadata+0x1e/0x46
 [<ffffffffa0230dab>] ? :ext4dev:ext4_free_inode+0x2b7/0x324
 [<ffffffffa0233db2>] ? :ext4dev:ext4_delete_inode+0xb7/0xd5
 [<ffffffffa0233cfb>] ? :ext4dev:ext4_delete_inode+0x0/0xd5
 [<ffffffff802ace23>] ? generic_delete_inode+0xab/0x11f
 [<ffffffff802ac358>] ? d_delete+0x49/0xb1
 [<ffffffff802a2e5c>] ? vfs_unlink+0xe3/0x102
 [<ffffffffa02b9b79>] ? :nfsd:nfsd_unlink+0x1e9/0x267
 [<ffffffffa02c17d8>] ? :nfsd:nfsd3_proc_remove+0x9d/0xaa
 [<ffffffffa02b6245>] ? :nfsd:nfsd_dispatch+0xde/0x1b6
 [<ffffffffa026b55b>] ? :sunrpc:svc_process+0x408/0x6e9
 [<ffffffff80429a04>] ? __down_read+0x12/0xa1
 [<ffffffffa02b667c>] ? :nfsd:nfsd+0x0/0x2a4
 [<ffffffffa02b6810>] ? :nfsd:nfsd+0x194/0x2a4
 [<ffffffff80230196>] ? schedule_tail+0x27/0x5c
 [<ffffffff8020cf28>] ? child_rip+0xa/0x12
 [<ffffffffa02b667c>] ? :nfsd:nfsd+0x0/0x2a4
 [<ffffffff8020cf1e>] ? child_rip+0x0/0x12

Code: 03 25 00 00 20 00 48 85 c0 75 f1 f0 0f ba 2b 15 19 c0 85 c0 75 e8 83 7d 10 00 75 19 c7 45 10 01 00 00 00 41 8b 45 08 85 c0 7f 04 <0f> 0b eb fe ff c8 41 89 45 08 48 39 55 28 75 11 83 7d 0c 02 75 
RIP  [<ffffffffa021e5f5>] :jbd2:jbd2_journal_dirty_metadata+0x5f/0xe3
 RSP <ffff810009543c90>
---[ end trace 1336f55a961cc4ae ]---

# dumpe2fs /dev/work/wdata 
dumpe2fs 1.41.3 (12-Oct-2008)
Filesystem volume name:   <none>
Last mounted on:          <not available>
Filesystem UUID:          05867e70-54e2-48bf-8c67-5439e98c5982
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem flags:         signed_directory_hash test_filesystem 
Default mount options:    (none)
Filesystem state:         clean with errors
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              917536
Block count:              939525120
Reserved block count:     46976256
Free blocks:              507628950
Free inodes:              619199
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      799
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         32
Inode blocks per group:   2
Flex block group size:    16
Filesystem created:       Thu Dec 11 10:36:48 2008
Last mount time:          Mon Jan 26 13:05:07 2009
Last write time:          Sat Jan 31 19:53:25 2009
Mount count:              1
Maximum mount count:      26
Last checked:             Mon Jan 26 12:53:17 2009
Check interval:           15552000 (6 months)
Next check after:         Sat Jul 25 13:53:17 2009
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:	          256
Required extra isize:     28
Desired extra isize:      28
Journal inode:            8
Default directory hash:   half_md4
Directory Hash Seed:      2714a303-cb2a-4bbc-8159-29bf52c617ca
Journal backup:           inode blocks
Journal size:             128M

(rest of dumpe2fs output omitted: 32MiB, can put it available somewhere).

t. Markus