[Linux-cluster] gfs over dm-multipath?

[孙俊伟]

Hi,all

	Can I create gfs directly on dm-multipath device(such as /dev/dm-0, /dev/dm-1 which is 
a partition on dm-0)? I've got some problem about it. I create a 2 nodes' cluster, 
and create a gfs on the device /dev/dm-1. the multipath config is:

[size=476 GB][features="1 queue_if_no_path"][hwhandler="0"]
\_ round-robin 0 [prio=2][active]
 \_ 0:0:0:0     sda           8:0   [active][ready]
 \_ 1:0:0:0     sdb           8:16  [active][ready]

Then, I start concurrent read/write/ls test. After about 40 hours, error occurs on one node.
The logs are as follows:

---->>the timestamp on the two nodes are not synchronized. they differs about 16.5 hours.

Dec 25 02:40:10 nd02 udev[8113]: udev_db.c: unable to read db file '/dev/.udevdb/class at vc@vcs1'
Dec 25 02:40:10 nd02 udev[8113]: udev_remove.c: 'vcs1' not found in database, falling back on default name
Dec 25 02:40:10 nd02 udev[8114]: udev_db.c: unable to read db file '/dev/.udevdb/class at vc@vcsa1'
Dec 25 02:40:10 nd02 udev[8114]: udev_remove.c: 'vcsa1' not found in database, falling back on default name
Dec 25 21:13:19 nd02 kernel: GFS: fsid=IPTV:dm1.0: jid=1: Trying to acquire journal lock...
Dec 25 21:13:19 nd02 kernel: GFS: fsid=IPTV:dm1.0: jid=1: Looking at journal...
Dec 25 21:13:21 nd02 kernel: GFS: fsid=IPTV:dm1.0: jid=1: Acquiring the transaction lock...
Dec 25 21:13:22 nd02 kernel: GFS: fsid=IPTV:dm1.0: jid=1: Replaying journal...
Dec 25 21:13:26 nd02 kernel: GFS: fsid=IPTV:dm1.0: jid=1: Replayed 166 of 327 blocks
Dec 25 21:13:26 nd02 kernel: GFS: fsid=IPTV:dm1.0: jid=1: replays = 166, skips = 2, sames = 159
Dec 25 21:13:26 nd02 kernel: GFS: fsid=IPTV:dm1.0: jid=1: Journal replayed in 6s
Dec 25 21:13:26 nd02 kernel: GFS: fsid=IPTV:dm1.0: jid=1: Done

---->>nd02 did something which would do when fencing, but no fence opertaion were done.

Dec 25 19:39:31 nd03 udev[7206]: udev_db.c: unable to read db file '/dev/.udevdb/class at vc@vcsa1'
Dec 25 19:39:31 nd03 udev[7205]: udev_db.c: unable to read db file '/dev/.udevdb/class at vc@vcs1'
Dec 25 19:39:31 nd03 udev[7206]: udev_remove.c: 'vcsa1' not found in database, falling back on default name
Dec 25 19:39:31 nd03 udev[7205]: udev_remove.c: 'vcs1' not found in database, falling back on default name
Dec 26 13:46:34 nd03 kernel: GFS: fsid=IPTV:dm1.1: fatal: filesystem consistency error
Dec 26 13:46:34 nd03 kernel: GFS: fsid=IPTV:dm1.1:   RG = 114708654
Dec 26 13:46:34 nd03 kernel: GFS: fsid=IPTV:dm1.1:   function = gfs_setbit
Dec 26 13:46:34 nd03 kernel: GFS: fsid=IPTV:dm1.1:   file = /home/sunjw/projects/cluster.STABLE/gfs-kernel/src/gfs/bits.c, li
ne = 71
Dec 26 13:46:34 nd03 kernel: GFS: fsid=IPTV:dm1.1:   time = 1135575994
Dec 26 13:46:34 nd03 kernel: GFS: fsid=IPTV:dm1.1: about to withdraw from the cluster
Dec 26 13:46:34 nd03 kernel: GFS: fsid=IPTV:dm1.1: waiting for outstanding I/O
Dec 26 13:46:35 nd03 kernel: GFS: fsid=IPTV:dm1.1: telling LM to withdraw
Dec 26 13:46:44 nd03 kernel: lock_dlm: withdraw abandoned memory
Dec 26 13:46:44 nd03 kernel: GFS: fsid=IPTV:dm1.1: withdrawn
                                                                

Thanks for any reply.
Luckey.