[Linux-cluster] kernel BUG at fs/gfs2/rgrp.c:1458!

[Peter Schobel]

We are experiencing fatal exceptions on a four node Linux cluster
using a gfs2 filesystem. Any help would be appreciated. Am happy to
provide additional info.

kernel BUG at fs/gfs2/rgrp.c:1458!
invalid opcode: 0000 [#1]
SMP
last sysfs file: /devices/pci0000:00/0000:00:00.0/irq
Modules linked in: ipv6 xfrm_nalgo crypto_api lock_dlm gfs2 dlm
configfs sunrpc dm_round_robin dm_multipath scsi_dh video hwmon
backlight sbs i2c_ec
i2c_cord
CPU:    0
EIP:    0060:[<f96e04da>]    Not tainted VLI
EFLAGS: 00010246   (2.6.18-128.1.16.el5PAE #1)
EIP is at gfs2_alloc_data+0x75/0x155 [gfs2]
eax: ffffffff   ebx: 00000000   ecx: 00000000   edx: 00000001
esi: 05ec1513   edi: 00000000   ebp: f51aa114   esp: edf96c74
ds: 007b   es: 007b   ss: 0068
Process p4v.bin (pid: 8895, ti=edf96000 task=ccfe0550 task.ti=edf96000)
Stack: d3b34548 f7242000 d9663380 00000000 d3b34548 00000000 d4575000 f96c4db2
       cf449378 d4575140 00001000 00000000 cf449378 d3b34548 edf96cf4 f96c50a0
       edf96cf4 00000001 edf96d18 edf96d10 00000000 0000000c 00000000 0000c000
Call Trace:
 [<f96c4db2>] lookup_block+0xb4/0x153 [gfs2]
 [<f96c50a0>] gfs2_block_map+0x24f/0x392 [gfs2]
 [<c047364e>] set_bh_page+0x43/0x4c
 [<c047371f>] alloc_page_buffers+0x74/0xba
 [<c0474455>] __block_prepare_write+0x1a2/0x439
 [<f96cc405>] do_promote+0xe8/0x10b [gfs2]
 [<c0474702>] block_prepare_write+0x16/0x23
 [<f96c4e51>] gfs2_block_map+0x0/0x392 [gfs2]
 [<f96d5005>] gfs2_write_begin+0x2af/0x359 [gfs2]
 [<f96c4e51>] gfs2_block_map+0x0/0x392 [gfs2]
 [<f96d6823>] gfs2_file_buffered_write+0x10d/0x287 [gfs2]
 [<c0428969>] current_fs_time+0x4a/0x55
 [<f96d6c71>] __gfs2_file_aio_write_nolock+0x2d4/0x32d [gfs2]
 [<c05ab197>] sock_aio_read+0x53/0x61
 [<f96d6e24>] gfs2_file_write_nolock+0xb0/0x111 [gfs2]
 [<c0434a97>] autoremove_wake_function+0x0/0x2d
 [<c0434a97>] autoremove_wake_function+0x0/0x2d
 [<f96d6f1b>] gfs2_file_write+0x0/0x94 [gfs2]
 [<f96d6f55>] gfs2_file_write+0x3a/0x94 [gfs2]
 [<f96d6f1b>] gfs2_file_write+0x0/0x94 [gfs2]
 [<c04720ef>] vfs_write+0xa1/0x143
 [<c04726e1>] sys_write+0x3c/0x63
 [<c0404ead>] sysenter_past_esp+0x56/0x79
 =======================
Code: 16 31 d2 01 f0 11 fa 39 d3 77 0c 72 04 39 c1 73 06 89 ca 29 f2
eb 03 8b 55 70 31 c9 89 e8 6a 01 e8 39 e8 ff ff 5a 83 f8 ff 75 08 <0f>
0b b2 05
7c 5d 6
EIP: [<f96e04da>] gfs2_alloc_data+0x75/0x155 [gfs2] SS:ESP 0068:edf96c74
 <0>Kernel panic - not syncing: Fatal exception


kernel BUG at fs/gfs2/rgrp.c:1458!
invalid opcode: 0000 [#1]
SMP
last sysfs file:
/devices/pci0000:00/0000:00:02.0/0000:04:00.0/0000:05:00.0/0000:06:00.0/0000:07:00.0/irq
Modules linked in: ipv6 xfrm_nalgo crypto_api lock_dlm gfs2 dlm
configfs sunrpc dm_round_robin dm_multipath scsi_dh video hwmon
backlight sbs i2c_ec i2c_cord
CPU:    1
EIP:    0060:[<f96df4da>]    Not tainted VLI
EFLAGS: 00010246   (2.6.18-128.1.16.el5PAE #1)
EIP is at gfs2_alloc_data+0x75/0x155 [gfs2]
eax: ffffffff   ebx: 00000000   ecx: 00000000   edx: 00000001
esi: 05ec1513   edi: 00000000   ebp: f0e11858   esp: f01f1c74
ds: 007b   es: 007b   ss: 0068
Process cp (pid: 31700, ti=f01f1000 task=f2e83550 task.ti=f01f1000)
Stack: f4361d68 f6cb1000 e45b7dc0 00000000 f4361d68 00000001 d25b8000 f96c3db2
       d12d53ac d25b8e10 00001000 00000001 e40686ec f4361d68 f01f1cf4 f96c40a0
       f01f1cf4 00000001 f01f1d18 f01f1d10 00000000 000013a5 00000000 013a5000
Call Trace:
 [<f96c3db2>] lookup_block+0xb4/0x153 [gfs2]
 [<f96c40a0>] gfs2_block_map+0x24f/0x392 [gfs2]
 [<c047364e>] set_bh_page+0x43/0x4c
 [<c047371f>] alloc_page_buffers+0x74/0xba
 [<c0474455>] __block_prepare_write+0x1a2/0x439
 [<f96cb405>] do_promote+0xe8/0x10b [gfs2]
 [<c0474702>] block_prepare_write+0x16/0x23
 [<f96c3e51>] gfs2_block_map+0x0/0x392 [gfs2]
 [<f96d4005>] gfs2_write_begin+0x2af/0x359 [gfs2]
 [<f96c3e51>] gfs2_block_map+0x0/0x392 [gfs2]
 [<f96d5823>] gfs2_file_buffered_write+0x10d/0x287 [gfs2]
 [<c0428969>] current_fs_time+0x4a/0x55
 [<f96d5c71>] __gfs2_file_aio_write_nolock+0x2d4/0x32d [gfs2]
 [<f96d5e24>] gfs2_file_write_nolock+0xb0/0x111 [gfs2]
 [<c0434a97>] autoremove_wake_function+0x0/0x2d
 [<c0434a97>] autoremove_wake_function+0x0/0x2d
 [<f96d5f1b>] gfs2_file_write+0x0/0x94 [gfs2]
 [<f96d5f55>] gfs2_file_write+0x3a/0x94 [gfs2]
 [<f96d5f1b>] gfs2_file_write+0x0/0x94 [gfs2]
 [<c04720ef>] vfs_write+0xa1/0x143
 [<c04726e1>] sys_write+0x3c/0x63
 [<c0404ead>] sysenter_past_esp+0x56/0x79
 =======================
Code: 16 31 d2 01 f0 11 fa 39 d3 77 0c 72 04 39 c1 73 06 89 ca 29 f2
eb 03 8b 55 70 31 c9 89 e8 6a 01 e8 39 e8 ff ff 5a 83 f8 ff 75 08 <0f>
0b b2 05 7c 4d 6
EIP: [<f96df4da>] gfs2_alloc_data+0x75/0x155 [gfs2] SS:ESP 0068:f01f1c74
 <0>Kernel panic - not syncing: Fatal exception

Thanks in advance,

-- 
Peter Schobel
~