<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:st1="urn:schemas-microsoft-com:office:smarttags" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv=Content-Type content="text/html; charset=us-ascii"> <meta name=Generator content="Microsoft Word 11 (filtered medium)"> <o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="PersonName"/>  <style>  </style> </head> <body lang=EN-US link=blue vlink=purple> <div class=Section1> Hello,<o:p></o:p> <o:p> </o:p> Just thought I’d plug in some more info I’ve also tried testing this with a stock FC4 client with all the Cluster RPMs installed.<o:p></o:p> <o:p> </o:p> GFS: fsid=sclients:mygfs.0: jid=17: Trying to acquire journal lock...<o:p></o:p> GFS: fsid=sclients:mygfs.0: jid=17: Looking at journal...<o:p></o:p> GFS: fsid=sclients:mygfs.0: jid=17: Done<o:p></o:p> GFS: fsid=sclients:mygfs.0: jid=18: Trying to acquire journal lock...<o:p></o:p> GFS: fsid=sclients:mygfs.0: jid=18: Looking at journal...<o:p></o:p> GFS: fsid=sclients:mygfs.0: jid=18: Done<o:p></o:p> GFS: fsid=sclients:mygfs.0: jid=19: Trying to acquire journal lock...<o:p></o:p> GFS: fsid=sclients:mygfs.0: jid=19: Looking at journal...<o:p></o:p> GFS: fsid=sclients:mygfs.0: jid=19: Done<o:p></o:p> GFS: fsid=sclients:mygfs.0: jid=20: Trying to acquire journal lock...<o:p></o:p> GFS: fsid=sclients:mygfs.0: jid=20: Looking at journal...<o:p></o:p> attempt to access beyond end of device<o:p></o:p> dm-0: rw=0, want=1146990600, limit=1146990592<o:p></o:p> GFS: fsid=sclients:mygfs.0: fatal: I/O error<o:p></o:p> GFS: fsid=sclients:mygfs.0: block = 143373824<o:p></o:p> GFS: fsid=sclients:mygfs.0: function = gfs_dreread<o:p></o:p> GFS: fsid=sclients:mygfs.0: file = /usr/src/build/588748-i686/BUILD/xen0/src/gfs/dio.c, line = 576<o:p></o:p> GFS: fsid=sclients:mygfs.0: time = 1123268277<o:p></o:p> GFS: fsid=sclients:mygfs.0: about to withdraw from the cluster<o:p></o:p> GFS: fsid=sclients:mygfs.0: waiting for outstanding I/O<o:p></o:p> GFS: fsid=sclients:mygfs.0: telling LM to withdraw<o:p></o:p> lock_dlm: withdraw abandoned memory<o:p></o:p> GFS: fsid=sclients:mygfs.0: withdrawn<o:p></o:p> GFS: fsid=sclients:mygfs.0: jid=20: Failed<o:p></o:p> GFS: fsid=sclients:mygfs.0: error recovering journal 20: -5<o:p></o:p> [root@5n@k3bi73 ~]# <o:p></o:p> Aug 5 11:57:57 5n@k3bi73 kernel: GFS: fsid=sclients:mygfs.0: time = 1123268277<o:p></o:p> Aug 5 11:57:57 5n@k3bi73 kernel: GFS: fsid=sclients:mygfs.0: about to withdraw from the cluster<o:p></o:p> Aug 5 11:57:57 5n@k3bi73 kernel: GFS: fsid=sclients:mygfs.0: waiting for outstanding I/O<o:p></o:p> Aug 5 11:57:57 5n@k3bi73 kernel: GFS: fsid=sclients:mygfs.0: telling LM to withdraw<o:p></o:p> Aug 5 11:57:57 5n@k3bi73 kernel: lock_dlm: withdraw abandoned memory<o:p></o:p> Aug 5 11:57:57 5n@k3bi73 kernel: GFS: fsid=sclients:mygfs.0: withdrawn<o:p></o:p> Aug 5 11:57:57 5n@k3bi73 kernel: GFS: fsid=sclients:mygfs.0: jid=20: Failed<o:p></o:p> <o:p> </o:p> This off a multipathed device which dmsetup status gives the output below:<o:p></o:p> <o:p> </o:p> dm-1: 0 1146990592 multipath 1 0 0 2 1 A 0 1 0 251:0 A 0 E 0 1 0 251:4 A 0<o:p></o:p> dmsetup deps gives<o:p></o:p> dm-1: 2 dependencies : (251, 4) (251, 0)<o:p></o:p> and dmsetup info gives<o:p></o:p> Name: dm-1<o:p></o:p> State: ACTIVE<o:p></o:p> Tables present: LIVE<o:p></o:p> Open count: 0<o:p></o:p> Event number: 0<o:p></o:p> Major, minor: 253, 1<o:p></o:p> Number of targets: 1<o:p></o:p> <o:p> </o:p> <o:p> </o:p> <div> <div class=MsoNormal align=center style='text-align:center'> <hr size=2 width="100%" align=center tabindex=-1> </div> From: brianu [mailto:brianu@silvercash.com] Sent: Friday, August 05, 2005 10:38 AM To: <st1:PersonName w:st="on">linux-cluster@redhat.com</st1:PersonName> Cc: brianu@silvercash.com Subject: RE: values for gnbd multipath dmsetup<o:p></o:p> </div> <o:p> </o:p> Hello,<o:p></o:p> <o:p> </o:p> Ok I figured out id just try some of the vaules from the previous post without fully understanding them, and multipath appears to be working.<o:p></o:p> <o:p> </o:p> dm-1<o:p></o:p> [size=546 GB][features="0"][hwhandler="0"] \_ round-robin 0 [active][first]<o:p></o:p> \_ 0:0:0:0 251:0 [undef ][active]<o:p></o:p> \_ round-robin 0 [enabled]<o:p></o:p> \_ 0:0:0:0 251:4 [undef ][active]<o:p></o:p> <o:p> </o:p> <o:p> </o:p> <o:p> </o:p> But I stil get an error<o:p></o:p> <o:p> </o:p> [root@dell-1650-31 ~]# mount -t gfs /dev/mapper/dm-1 /mnt/gfs1<o:p></o:p> mount: /dev/dm-1: can't read superblock<o:p></o:p> <o:p> </o:p> if I do a dmsetup remove dm-1, then mount the individual gnbds all is well, but the purpose of this is to enable some sore of failover which I am told GNBD has the capability of doing.<o:p></o:p> <o:p> </o:p> From redhats main site and documentation for gfs 6.1 they state that multipath is not supported in the 6.1 realease however I optained this source from CVS and the main docs for <a href="http://sources.redhat.com/cluster/gnbd/">http://sources.redhat.com/cluster/gnbd/</a> state that multipath is an option. Can someone clarify whether the CVS stabile sources for kernel-2.6.12 is multipath compatable, or am I doing something wrong?<o:p></o:p> <o:p> </o:p> Current specs.<o:p></o:p> SAN -> MSA-1000<o:p></o:p> <o:p> </o:p> 3 GNBD servers currently using software iSCSI to mount that SAN – will prob go fiber if I can figure this out. ( lets say this cluster is called cluster1)<o:p></o:p> Using DLM & GNBD<o:p></o:p> <o:p> </o:p> 1 client for testing separate cluster name lets say “cluster2”<o:p></o:p> Client mounted the gnbd from one of the servers that is exporting it, the servers are not mounting it, then formatted the device with gfs & created 20 journals size of 32MB each, remounted the device and verified write and read (bonnie++)<o:p></o:p> Ran dmsetup to round robin the devices then failed to mount the volume as shown above.<o:p></o:p> <o:p> </o:p> <o:p> </o:p> <o:p> </o:p> Brian Urrutia<o:p></o:p> System Administrator<o:p></o:p> Price Communications Inc.<o:p></o:p> <o:p> </o:p> </div> </body> </html>