[Linux-cluster] problem exporting GFS2 filesystem with NFS
Wendy Cheng
wcheng at redhat.com
Tue Jan 23 18:40:57 UTC 2007
Benoit DUFFAU wrote:
> Hello all,
>
> I've (tried to) set up a two-nodes cluster using the lattest kernel
> (2.6.19.2) with GFS2 included in the kernel.
>
> Everything went fine using the doc found in
>
> http://sources.redhat.com/cgi-bin/cvsweb.cgi/cluster/doc/usage.txt?rev=1.35&content-type=text/x-cvsweb-markup&cvsroot=cluster
>
> so i did everything discribed to bring the cluster on.
>
> The problem occurs when i try to export the gfs2 mounted file system
> with NFS.
>
GFS2 used to have a special version of "filldir" routine but got removed
due to lkml code review. Look like that (new) changes have issues. If
you have some spare time, please open a bugzilla. We do have a rough
idea what goes wrong - will find a solution to this issue soon.
-- Wendy
> I use a debian stable (sarge) and the nfs-kernel-server package that
> uses the nfsd kernel module
>
> I export the repository (let's say /test) , I mount it on the nfs client
> machine (let's say on /mnt for exemple) , I issue a ls -l /mnt (on the
> client) and the server crashes.
>
> I attach the syslog part that includes the following commands :
>
> 1-mounting the gfs2 filesystem localy on the NFS serveur
> 2-mounting the exported directory (/test) on the machine named client1
> 3-issuing a "ls -l" on the client machine.
>
> I also use on this machine Vmware and drbd 0.8rc2 but i did exactly the
> same test on the other node with a untainted kernel since i do not use
> vmware on the other node, and it crashes the same way ...
>
> looks like if it was impossible to export gfs2 over NFS ?
> Incompatibilities between lock methods ?
>
> does someone already notice this ?
>
> Benoit DUFFAU
>
> +---------------------------------------------------------------------+
> Combining consulting and technology solutions offers enables Devoteam
> to provide its customers with independent advice and effective solutions
> that meet their strategic objectives (IT performance and optimisation)
> in complementary areas: networks, systems infrastructure, security
> and e-business applications.
> Created in 1995, Devoteam achieved in 2005 a turnover of 199 million euros
> and an operating margin of 7%. The group counts 2,400 employees through
> sixteen countries in Europe, the Middle East and North Africa.
> Listed on Euronext (Eurolist B compartment) since October 28, 1999.
> Part of the Nexteconomy, CAC SMALL 90, IT CAC 50, SBF 250 index of
> Euronext Paris
> ISIN: FR 000007379 3, Reuters: DVTM.LM, Bloomberg: DEVO FP
> +---------------------------------------------------------------------+
>
> ------------------------------------------------------------------------
>
> Jan 23 15:40:35 ftp kernel: GFS2: fsid=: Trying to join cluster "lock_dlm", "sympa:sympafs"
> Jan 23 15:40:35 ftp kernel: dlm: sympafs: recover 1
> Jan 23 15:40:35 ftp kernel: GFS2: fsid=sympa:sympafs.0: Joined cluster. Now mounting FS...
> Jan 23 15:40:35 ftp kernel: dlm: sympafs: add member 2
> Jan 23 15:40:35 ftp kernel: dlm: sympafs: total members 1 error 0
> Jan 23 15:40:35 ftp kernel: dlm: sympafs: dlm_recover_directory
> Jan 23 15:40:35 ftp kernel: dlm: sympafs: dlm_recover_directory 0 entries
> Jan 23 15:40:35 ftp kernel: dlm: sympafs: recover 1 done: 40 ms
> Jan 23 15:40:35 ftp kernel: GFS2: fsid=sympa:sympafs.0: jid=0, already locked for use
> Jan 23 15:40:35 ftp kernel: GFS2: fsid=sympa:sympafs.0: jid=0: Looking at journal...
> Jan 23 15:40:36 ftp kernel: GFS2: fsid=sympa:sympafs.0: jid=0: Done
> Jan 23 15:40:36 ftp kernel: GFS2: fsid=sympa:sympafs.0: jid=1: Trying to acquire journal lock...
> Jan 23 15:40:36 ftp kernel: GFS2: fsid=sympa:sympafs.0: jid=1: Looking at journal...
> Jan 23 15:40:36 ftp kernel: GFS2: fsid=sympa:sympafs.0: jid=1: Done
> Jan 23 15:43:27 ftp rpc.mountd: authenticated mount request from client1:694 for /test (/test)
> Jan 23 15:43:46 ftp kernel: original: gfs2_glock_nq_atime+0x17a/0x31a
> Jan 23 15:43:46 ftp kernel: pid : 10025
> Jan 23 15:43:47 ftp kernel: lock type : 2 lock state : 1
> Jan 23 15:43:47 ftp kernel: new: gfs2_getattr+0x33/0x6c
> Jan 23 15:43:47 ftp kernel: pid : 10025
> Jan 23 15:43:47 ftp kernel: lock type : 2 lock state : 1
> Jan 23 15:43:48 ftp kernel: ------------[ cut here ]------------
> Jan 23 15:43:48 ftp kernel: kernel BUG at fs/gfs2/glock.c:1193!
> Jan 23 15:43:48 ftp kernel: invalid opcode: 0000 [#1]
> Jan 23 15:43:48 ftp kernel: SMP
> Jan 23 15:43:48 ftp kernel: Modules linked in: nfsd exportfs lockd nfs_acl sunrpc vmnet(P) vmmon(P) sctp drbd
> Jan 23 15:43:48 ftp kernel: CPU: 1
> Jan 23 15:43:48 ftp kernel: EIP: 0060:[add_to_queue+221/373] Tainted: P VLI
> Jan 23 15:43:48 ftp kernel: EFLAGS: 00010286 (2.6.19.2 #3)
> Jan 23 15:43:48 ftp kernel: EIP is at add_to_queue+0xdd/0x175
> Jan 23 15:43:48 ftp kernel: eax: 00000020 ebx: d7945e68 ecx: d7945a38 edx: c0512440
> Jan 23 15:43:48 ftp kernel: esi: d543b9dc edi: d7945a78 ebp: c8b9e000 esp: d7945a34
> Jan 23 15:43:48 ftp kernel: ds: 007b es: 007b ss: 0068
> Jan 23 15:43:48 ftp kernel: Process nfsd (pid: 10025, ti=d7944000 task=da45f550 task.ti=d7944000)
> Jan 23 15:43:48 ftp kernel: Stack: c0512440 00000002 00000001 d543b9dc d7945a78 00000000 c023021b d7945a78
> Jan 23 15:43:48 ftp kernel: d7945a78 d7945b00 d0bf51a8 d7945c78 c023f7bc d7945a78 00000003 00000008
> Jan 23 15:43:48 ftp kernel: d7945a78 d7945a78 d7945a78 d543b9dc da45f550 00000003 00000008 00000000
> Jan 23 15:43:48 ftp kernel: Call Trace:
> Jan 23 15:43:48 ftp kernel: [gfs2_glock_nq+62/150] gfs2_glock_nq+0x3e/0x96
> Jan 23 15:43:48 ftp kernel: [gfs2_getattr+59/108] gfs2_getattr+0x3b/0x6c
> Jan 23 15:43:48 ftp kernel: [gfs2_getattr+51/108] gfs2_getattr+0x33/0x6c
> Jan 23 15:43:48 ftp kernel: [vfs_getattr+47/141] vfs_getattr+0x2f/0x8d
> Jan 23 15:43:48 ftp kernel: [pg0+948462326/1066693632] encode_post_op_attr+0x4a/0x238 [nfsd]
> Jan 23 15:43:48 ftp kernel: [pg0+948424111/1066693632] fh_compose+0x1da/0x419 [nfsd]
> Jan 23 15:43:48 ftp kernel: [smp_send_reschedule+30/34] smp_send_reschedule+0x1e/0x22
> Jan 23 15:43:48 ftp kernel: [dput+34/291] dput+0x22/0x123
> Jan 23 15:43:48 ftp kernel: [pg0+948472018/1066693632] compose_entry_fh+0x10b/0x115 [nfsd]
> Jan 23 15:43:48 ftp kernel: [pg0+948472571/1066693632] encode_entry+0x21f/0x536 [nfsd]
> Jan 23 15:43:48 ftp kernel: [activate_task+100/174] activate_task+0x64/0xae
> Jan 23 15:43:48 ftp kernel: [smp_send_reschedule+30/34] smp_send_reschedule+0x1e/0x22
> Jan 23 15:43:48 ftp kernel: [try_to_wake_up+657/668] try_to_wake_up+0x291/0x29c
> Jan 23 15:43:48 ftp kernel: [__next_cpu+32/47] __next_cpu+0x20/0x2f
> Jan 23 15:43:48 ftp kernel: [find_busiest_group+391/1168] find_busiest_group+0x187/0x490
> Jan 23 15:43:48 ftp kernel: [gfs2_glmutex_lock+142/148] gfs2_glmutex_lock+0x8e/0x94
> Jan 23 15:43:48 ftp kernel: [filldir_func+80/211] filldir_func+0x50/0xd3
> Jan 23 15:43:48 ftp kernel: [do_filldir_main+430/494] do_filldir_main+0x1ae/0x1ee
> Jan 23 15:43:48 ftp kernel: [filldir_func+0/211] filldir_func+0x0/0xd3
> Jan 23 15:43:48 ftp kernel: [gfs2_dir_read+362/408] gfs2_dir_read+0x16a/0x198
> Jan 23 15:43:48 ftp kernel: [filldir_func+0/211] filldir_func+0x0/0xd3
> Jan 23 15:43:48 ftp kernel: [gfs2_readdir+173/207] gfs2_readdir+0xad/0xcf
> Jan 23 15:43:48 ftp kernel: [filldir_func+0/211] filldir_func+0x0/0xd3
> Jan 23 15:43:48 ftp kernel: [gfs2_glock_nq_atime+378/794] gfs2_glock_nq_atime+0x17a/0x31a
> Jan 23 15:43:48 ftp kernel: [dentry_open+88/94] dentry_open+0x58/0x5e
> Jan 23 15:43:48 ftp kernel: [pg0+948473401/1066693632] nfs3svc_encode_entry_plus+0x0/0x27 [nfsd]
> Jan 23 15:43:48 ftp kernel: [vfs_readdir+81/124] vfs_readdir+0x51/0x7c
> Jan 23 15:43:48 ftp kernel: [pg0+948473401/1066693632] nfs3svc_encode_entry_plus+0x0/0x27 [nfsd]
> Jan 23 15:43:48 ftp kernel: [pg0+948437677/1066693632] nfsd_readdir+0x95/0x107 [nfsd]
> Jan 23 15:43:48 ftp kernel: [pg0+948473401/1066693632] nfs3svc_encode_entry_plus+0x0/0x27 [nfsd]
> Jan 23 15:43:48 ftp kernel: [pg0+948461192/1066693632] nfsd3_proc_readdirplus+0x110/0x1ee [nfsd]
> Jan 23 15:43:48 ftp kernel: [pg0+948473401/1066693632] nfs3svc_encode_entry_plus+0x0/0x27 [nfsd]
> Jan 23 15:43:48 ftp kernel: [pg0+948469685/1066693632] nfs3svc_decode_readdirplusargs+0x107/0x145 [nfsd]
> Jan 23 15:43:48 ftp kernel: [pg0+948414565/1066693632] nfsd_dispatch+0xed/0x1d8 [nfsd]
> Jan 23 15:43:48 ftp kernel: [pg0+948183442/1066693632] svc_process+0x375/0x5de [sunrpc]
> Jan 23 15:43:48 ftp kernel: [pg0+948414052/1066693632] nfsd+0x19c/0x2b0 [nfsd]
> Jan 23 15:43:48 ftp kernel: [pg0+948413640/1066693632] nfsd+0x0/0x2b0 [nfsd]
> Jan 23 15:43:48 ftp kernel: [kernel_thread_helper+7/16] kernel_thread_helper+0x7/0x10
> Jan 23 15:43:48 ftp kernel: =======================
> Jan 23 15:43:48 ftp kernel: Code: 00 c7 04 24 9a ba 4f c0 89 44 24 04 e8 62 b5 ee ff 8b 46 20 89 44 24 08 8b 46 14 c7 04 24 40 24 51 c0 89 44 24 04 e8 48 b5 ee ff <0f> 0b a9 04 e8 b9 4f c0 8b 56 48 8d 4e 48 8b 5f 0c 8b 02 0f 18
> Jan 23 15:43:48 ftp kernel: EIP: [add_to_queue+221/373] add_to_queue+0xdd/0x175 SS:ESP 0068:d7945a34
>
>
> ------------------------------------------------------------------------
>
> --
> Linux-cluster mailing list
> Linux-cluster at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster
More information about the Linux-cluster
mailing list