[Linux-cluster] gfs2 partition withdrawn

[Nicolas Ferré]

Hi,

We have a problem with our cluster, a gfs2 fs cannot be accessed some 
times after the system reboot. I have to manually umount/mount it.

Here is the relevant part of /var/log/messages:
Oct  3 11:46:14 slater kernel: GFS2: fsid=crcmm:home.1: fatal: invalid 
metadata block
Oct  3 11:46:14 slater kernel: GFS2: fsid=crcmm:home.1:   bh = 114419123 
(magic number)
Oct  3 11:46:14 slater kernel: GFS2: fsid=crcmm:home.1:   function = 
gfs2_meta_indirect_buffer, file = fs/gfs2/meta_io.c, line = 334
Oct  3 11:46:14 slater kernel: GFS2: fsid=crcmm:home.1: about to 
withdraw this file system
Oct  3 11:46:14 slater kernel: GFS2: fsid=crcmm:home.1: telling LM to 
withdraw
Oct  3 11:46:16 slater kernel: VFS:Filesystem freeze failed
Oct  3 11:46:22 slater snmpd[7344]: Connection from UDP: [127.0.0.1]:58640
Oct  3 11:46:22 slater snmpd[7344]: Received SNMP packet(s) from UDP: 
[127.0.0.1]:58640
Oct  3 11:46:37 slater snmpd[7344]: Connection from UDP: [127.0.0.1]:47125
Oct  3 11:46:37 slater snmpd[7344]: Received SNMP packet(s) from UDP: 
[127.0.0.1]:47125
Oct  3 11:46:53 slater snmpd[7344]: Connection from UDP: [127.0.0.1]:33910
Oct  3 11:46:53 slater snmpd[7344]: Received SNMP packet(s) from UDP: 
[127.0.0.1]:33910
Oct  3 11:46:53 slater kernel: dlm: home: group leave failed -512 0
Oct  3 11:46:53 slater kernel: GFS2: fsid=crcmm:home.1: withdrawn
Oct  3 11:46:53 slater kernel:
Oct  3 11:46:53 slater kernel: Call Trace:
Oct  3 11:46:53 slater kernel:  [<ffffffff8863a3ce>] 
:gfs2:gfs2_lm_withdraw+0xc1/0xd0
Oct  3 11:46:53 slater kernel:  [<ffffffff80063a06>] __wait_on_bit+0x60/0x6e
Oct  3 11:46:53 slater kernel:  [<ffffffff800153ac>] sync_buffer+0x0/0x3f
Oct  3 11:46:53 slater kernel:  [<ffffffff80063a80>] 
out_of_line_wait_on_bit+0x6c/0x78
Oct  3 11:46:53 slater kernel:  [<ffffffff8009f6ef>] 
wake_bit_function+0x0/0x23
Oct  3 11:46:53 slater kernel:  [<ffffffff8001a7ac>] submit_bh+0x10a/0x111
Oct  3 11:46:53 slater kernel:  [<ffffffff8864d547>] 
:gfs2:gfs2_meta_check_ii+0x2c/0x38
Oct  3 11:46:53 slater kernel:  [<ffffffff8863de01>] 
:gfs2:gfs2_meta_indirect_buffer+0x104/0x15f
Oct  3 11:46:53 slater kernel:  [<ffffffff8863d993>] 
:gfs2:gfs2_getbuf+0x106/0x115
Oct  3 11:46:53 slater kernel:  [<ffffffff8862e786>] 
:gfs2:recursive_scan+0x96/0x175
Oct  3 11:46:53 slater kernel:  [<ffffffff8862e82c>] 
:gfs2:recursive_scan+0x13c/0x175
Oct  3 11:46:53 slater kernel:  [<ffffffff8862f6bc>] 
:gfs2:do_strip+0x0/0x349
Oct  3 11:46:53 slater kernel:  [<ffffffff8862e8fe>] 
:gfs2:trunc_dealloc+0x99/0xe7
Oct  3 11:46:53 slater kernel:  [<ffffffff8862f6bc>] 
:gfs2:do_strip+0x0/0x349
Oct  3 11:46:53 slater kernel:  [<ffffffff88645dd2>] 
:gfs2:gfs2_delete_inode+0xdd/0x191
Oct  3 11:46:53 slater kernel:  [<ffffffff88645d3b>] 
:gfs2:gfs2_delete_inode+0x46/0x191
Oct  3 11:46:53 slater kernel:  [<ffffffff88635e77>] 
:gfs2:gfs2_glock_schedule_for_reclaim+0x5d/0x9a
Oct  3 11:46:53 slater kernel:  [<ffffffff88645cf5>] 
:gfs2:gfs2_delete_inode+0x0/0x191
Oct  3 11:46:53 slater kernel:  [<ffffffff8002f49e>] 
generic_delete_inode+0xc6/0x143
Oct  3 11:46:53 slater kernel:  [<ffffffff8864a99c>] 
:gfs2:gfs2_inplace_reserve_i+0x63b/0x691
Oct  3 11:46:53 slater kernel:  [<ffffffff80021f3f>] __up_read+0x19/0x7f
Oct  3 11:46:53 slater kernel:  [<ffffffff88635dd8>] 
:gfs2:do_promote+0xf5/0x137
Oct  3 11:46:53 slater kernel:  [<ffffffff8863f24a>] 
:gfs2:gfs2_write_begin+0x16c/0x339
Oct  3 11:46:53 slater kernel:  [<ffffffff88640a7b>] 
:gfs2:gfs2_file_buffered_write+0xf3/0x26c
Oct  3 11:46:53 slater kernel:  [<ffffffff88640e4c>] 
:gfs2:__gfs2_file_aio_write_nolock+0x258/0x28f
Oct  3 11:46:53 slater kernel:  [<ffffffff88640fee>] 
:gfs2:gfs2_file_write_nolock+0xaa/0x10f
Oct  3 11:46:54 slater kernel:  [<ffffffff800c5145>] 
generic_file_read+0xac/0xc5
Oct  3 11:46:54 slater kernel:  [<ffffffff8009f6c1>] 
autoremove_wake_function+0x0/0x2e
Oct  3 11:46:54 slater kernel:  [<ffffffff88635e77>] 
:gfs2:gfs2_glock_schedule_for_reclaim+0x5d/0x9a
Oct  3 11:46:54 slater kernel:  [<ffffffff8009f6c1>] 
autoremove_wake_function+0x0/0x2e
Oct  3 11:46:54 slater kernel:  [<ffffffff8864113e>] 
:gfs2:gfs2_file_write+0x49/0xa7
Oct  3 11:46:54 slater kernel:  [<ffffffff80016927>] vfs_write+0xce/0x174
Oct  3 11:46:54 slater kernel:  [<ffffffff800171df>] sys_write+0x45/0x6e
Oct  3 11:46:54 slater kernel:  [<ffffffff8006149d>] 
sysenter_do_call+0x1e/0x6a
Oct  3 11:46:54 slater kernel:
Oct  3 11:46:54 slater kernel: GFS2: fsid=crcmm:home.1: 
gfs2_delete_inode: -5

Can someone explain the meaning of such messages? And how to cure the 
problem ...

Regards,

-- 
Nicolas Ferre'
Laboratoire Chimie Provence
Universite' de Provence - France
Tel: +33 491282733
http://sites.univ-provence.fr/lcp-ct