[Linux-cluster] problem exporting GFS2 filesystem with NFS

Benoit DUFFAU benoit.duffau at devoteam.com
Tue Jan 23 16:05:47 UTC 2007 

Hello all,

I've (tried to) set up a two-nodes cluster using the lattest kernel
(2.6.19.2) with GFS2 included in the kernel.

Everything went fine using the doc found in 

http://sources.redhat.com/cgi-bin/cvsweb.cgi/cluster/doc/usage.txt?rev=1.35&content-type=text/x-cvsweb-markup&cvsroot=cluster

so i did everything discribed to bring the cluster on.

The problem occurs when i try to export the gfs2 mounted file system
with NFS.

I use a debian stable (sarge) and the nfs-kernel-server package that
uses the nfsd kernel module

I export the repository (let's say /test) , I mount it on the nfs client
machine (let's say on /mnt for exemple) , I issue a ls -l /mnt (on the
client) and the server crashes.

I attach the syslog part that includes the following commands : 

1-mounting the gfs2 filesystem localy on the NFS serveur
2-mounting the exported directory (/test) on the machine named client1
3-issuing a "ls -l" on the client machine.

I also use on this machine Vmware and drbd 0.8rc2 but i did exactly the
same test on the other node with a untainted kernel since i do not use
vmware on the other node, and it crashes the same way ...

looks like if it was impossible to export gfs2 over NFS ?
Incompatibilities between lock methods ? 

does someone already notice this ?

Benoit DUFFAU

+---------------------------------------------------------------------+
Combining consulting and technology solutions offers enables Devoteam
to provide its customers with independent advice and effective solutions
that meet their strategic objectives (IT performance and optimisation)
in complementary areas: networks, systems infrastructure, security
and e-business applications.
Created in 1995, Devoteam achieved in 2005 a turnover of 199 million euros
and an operating margin of 7%. The group counts 2,400 employees through
sixteen countries in Europe, the Middle East and North Africa.
Listed on Euronext (Eurolist B compartment) since October 28, 1999.
Part of the Nexteconomy, CAC SMALL 90, IT CAC 50, SBF 250 index of 
Euronext Paris
ISIN: FR 000007379 3, Reuters: DVTM.LM, Bloomberg: DEVO FP
+---------------------------------------------------------------------+
-------------- next part --------------
Jan 23 15:40:35 ftp kernel: GFS2: fsid=: Trying to join cluster "lock_dlm", "sympa:sympafs"
Jan 23 15:40:35 ftp kernel: dlm: sympafs: recover 1
Jan 23 15:40:35 ftp kernel: GFS2: fsid=sympa:sympafs.0: Joined cluster. Now mounting FS...
Jan 23 15:40:35 ftp kernel: dlm: sympafs: add member 2
Jan 23 15:40:35 ftp kernel: dlm: sympafs: total members 1 error 0
Jan 23 15:40:35 ftp kernel: dlm: sympafs: dlm_recover_directory 
Jan 23 15:40:35 ftp kernel: dlm: sympafs: dlm_recover_directory 0 entries
Jan 23 15:40:35 ftp kernel: dlm: sympafs: recover 1 done: 40 ms 
Jan 23 15:40:35 ftp kernel: GFS2: fsid=sympa:sympafs.0: jid=0, already locked for use
Jan 23 15:40:35 ftp kernel: GFS2: fsid=sympa:sympafs.0: jid=0: Looking at journal...
Jan 23 15:40:36 ftp kernel: GFS2: fsid=sympa:sympafs.0: jid=0: Done
Jan 23 15:40:36 ftp kernel: GFS2: fsid=sympa:sympafs.0: jid=1: Trying to acquire journal lock...
Jan 23 15:40:36 ftp kernel: GFS2: fsid=sympa:sympafs.0: jid=1: Looking at journal...
Jan 23 15:40:36 ftp kernel: GFS2: fsid=sympa:sympafs.0: jid=1: Done
Jan 23 15:43:27 ftp rpc.mountd: authenticated mount request from client1:694 for /test (/test)
Jan 23 15:43:46 ftp kernel: original: gfs2_glock_nq_atime+0x17a/0x31a
Jan 23 15:43:46 ftp kernel: pid : 10025
Jan 23 15:43:47 ftp kernel: lock type : 2 lock state : 1
Jan 23 15:43:47 ftp kernel: new: gfs2_getattr+0x33/0x6c
Jan 23 15:43:47 ftp kernel: pid : 10025
Jan 23 15:43:47 ftp kernel: lock type : 2 lock state : 1
Jan 23 15:43:48 ftp kernel: ------------[ cut here ]------------
Jan 23 15:43:48 ftp kernel: kernel BUG at fs/gfs2/glock.c:1193!
Jan 23 15:43:48 ftp kernel: invalid opcode: 0000 [#1]
Jan 23 15:43:48 ftp kernel: SMP 
Jan 23 15:43:48 ftp kernel: Modules linked in: nfsd exportfs lockd nfs_acl sunrpc vmnet(P) vmmon(P) sctp drbd
Jan 23 15:43:48 ftp kernel: CPU:    1
Jan 23 15:43:48 ftp kernel: EIP:    0060:[add_to_queue+221/373]    Tainted: P      VLI
Jan 23 15:43:48 ftp kernel: EFLAGS: 00010286   (2.6.19.2 #3)
Jan 23 15:43:48 ftp kernel: EIP is at add_to_queue+0xdd/0x175
Jan 23 15:43:48 ftp kernel: eax: 00000020   ebx: d7945e68   ecx: d7945a38   edx: c0512440
Jan 23 15:43:48 ftp kernel: esi: d543b9dc   edi: d7945a78   ebp: c8b9e000   esp: d7945a34
Jan 23 15:43:48 ftp kernel: ds: 007b   es: 007b   ss: 0068
Jan 23 15:43:48 ftp kernel: Process nfsd (pid: 10025, ti=d7944000 task=da45f550 task.ti=d7944000)
Jan 23 15:43:48 ftp kernel: Stack: c0512440 00000002 00000001 d543b9dc d7945a78 00000000 c023021b d7945a78
Jan 23 15:43:48 ftp kernel:        d7945a78 d7945b00 d0bf51a8 d7945c78 c023f7bc d7945a78 00000003 00000008
Jan 23 15:43:48 ftp kernel:        d7945a78 d7945a78 d7945a78 d543b9dc da45f550 00000003 00000008 00000000
Jan 23 15:43:48 ftp kernel: Call Trace:
Jan 23 15:43:48 ftp kernel:  [gfs2_glock_nq+62/150] gfs2_glock_nq+0x3e/0x96
Jan 23 15:43:48 ftp kernel:  [gfs2_getattr+59/108] gfs2_getattr+0x3b/0x6c
Jan 23 15:43:48 ftp kernel:  [gfs2_getattr+51/108] gfs2_getattr+0x33/0x6c
Jan 23 15:43:48 ftp kernel:  [vfs_getattr+47/141] vfs_getattr+0x2f/0x8d
Jan 23 15:43:48 ftp kernel:  [pg0+948462326/1066693632] encode_post_op_attr+0x4a/0x238 [nfsd]
Jan 23 15:43:48 ftp kernel:  [pg0+948424111/1066693632] fh_compose+0x1da/0x419 [nfsd]
Jan 23 15:43:48 ftp kernel:  [smp_send_reschedule+30/34] smp_send_reschedule+0x1e/0x22
Jan 23 15:43:48 ftp kernel:  [dput+34/291] dput+0x22/0x123
Jan 23 15:43:48 ftp kernel:  [pg0+948472018/1066693632] compose_entry_fh+0x10b/0x115 [nfsd]
Jan 23 15:43:48 ftp kernel:  [pg0+948472571/1066693632] encode_entry+0x21f/0x536 [nfsd]
Jan 23 15:43:48 ftp kernel:  [activate_task+100/174] activate_task+0x64/0xae
Jan 23 15:43:48 ftp kernel:  [smp_send_reschedule+30/34] smp_send_reschedule+0x1e/0x22
Jan 23 15:43:48 ftp kernel:  [try_to_wake_up+657/668] try_to_wake_up+0x291/0x29c
Jan 23 15:43:48 ftp kernel:  [__next_cpu+32/47] __next_cpu+0x20/0x2f
Jan 23 15:43:48 ftp kernel:  [find_busiest_group+391/1168] find_busiest_group+0x187/0x490
Jan 23 15:43:48 ftp kernel:  [gfs2_glmutex_lock+142/148] gfs2_glmutex_lock+0x8e/0x94
Jan 23 15:43:48 ftp kernel:  [filldir_func+80/211] filldir_func+0x50/0xd3
Jan 23 15:43:48 ftp kernel:  [do_filldir_main+430/494] do_filldir_main+0x1ae/0x1ee
Jan 23 15:43:48 ftp kernel:  [filldir_func+0/211] filldir_func+0x0/0xd3
Jan 23 15:43:48 ftp kernel:  [gfs2_dir_read+362/408] gfs2_dir_read+0x16a/0x198
Jan 23 15:43:48 ftp kernel:  [filldir_func+0/211] filldir_func+0x0/0xd3
Jan 23 15:43:48 ftp kernel:  [gfs2_readdir+173/207] gfs2_readdir+0xad/0xcf
Jan 23 15:43:48 ftp kernel:  [filldir_func+0/211] filldir_func+0x0/0xd3
Jan 23 15:43:48 ftp kernel:  [gfs2_glock_nq_atime+378/794] gfs2_glock_nq_atime+0x17a/0x31a
Jan 23 15:43:48 ftp kernel:  [dentry_open+88/94] dentry_open+0x58/0x5e
Jan 23 15:43:48 ftp kernel:  [pg0+948473401/1066693632] nfs3svc_encode_entry_plus+0x0/0x27 [nfsd]
Jan 23 15:43:48 ftp kernel:  [vfs_readdir+81/124] vfs_readdir+0x51/0x7c
Jan 23 15:43:48 ftp kernel:  [pg0+948473401/1066693632] nfs3svc_encode_entry_plus+0x0/0x27 [nfsd]
Jan 23 15:43:48 ftp kernel:  [pg0+948437677/1066693632] nfsd_readdir+0x95/0x107 [nfsd]
Jan 23 15:43:48 ftp kernel:  [pg0+948473401/1066693632] nfs3svc_encode_entry_plus+0x0/0x27 [nfsd]
Jan 23 15:43:48 ftp kernel:  [pg0+948461192/1066693632] nfsd3_proc_readdirplus+0x110/0x1ee [nfsd]
Jan 23 15:43:48 ftp kernel:  [pg0+948473401/1066693632] nfs3svc_encode_entry_plus+0x0/0x27 [nfsd]
Jan 23 15:43:48 ftp kernel:  [pg0+948469685/1066693632] nfs3svc_decode_readdirplusargs+0x107/0x145 [nfsd]
Jan 23 15:43:48 ftp kernel:  [pg0+948414565/1066693632] nfsd_dispatch+0xed/0x1d8 [nfsd]
Jan 23 15:43:48 ftp kernel:  [pg0+948183442/1066693632] svc_process+0x375/0x5de [sunrpc]
Jan 23 15:43:48 ftp kernel:  [pg0+948414052/1066693632] nfsd+0x19c/0x2b0 [nfsd]
Jan 23 15:43:48 ftp kernel:  [pg0+948413640/1066693632] nfsd+0x0/0x2b0 [nfsd]
Jan 23 15:43:48 ftp kernel:  [kernel_thread_helper+7/16] kernel_thread_helper+0x7/0x10
Jan 23 15:43:48 ftp kernel:  =======================
Jan 23 15:43:48 ftp kernel: Code: 00 c7 04 24 9a ba 4f c0 89 44 24 04 e8 62 b5 ee ff 8b 46 20 89 44 24 08 8b 46 14 c7 04 24 40 24 51 c0 89 44 24 04 e8 48 b5 ee ff <0f> 0b a9 04 e8 b9 4f c0 8b 56 48 8d 4e 48 8b 5f 0c 8b 02 0f 18
Jan 23 15:43:48 ftp kernel: EIP: [add_to_queue+221/373] add_to_queue+0xdd/0x175 SS:ESP 0068:d7945a34

More information about the Linux-cluster mailing list