[Linux-cluster] GFS Panic

[Derek Anderson]

On Friday 29 October 2004 09:32, Ritesh Agrawal wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> Hi All,
>
> ~ I have a problem related to GFS deployment on a SAN volume,
>
> ~ SCENARIO:
>
> ~ Clustered Mail Server application
>
> ~ Environment:
> ~ RedHat ES 3.0
> ~ Redhat Cluster Suite
> ~ Cluster file system - GFS (ver 6.0) on ex3fs

Um.  Can you elaborate on this a bit?  GFS ext3?

>
> ~ DELL PowerEdge 1550 / 2650 / 1650 / 1750
> ~ EMC CX series
> ~ HBA Qlogix 23xx
>
> ~ SET-UP
> ~ DNS--> load balancer(s) --> cluster node + file lock management --> SAN
> ~ (mutli LUN)
> ~ single file lock manager in the cluster (one among the cluster nodes)
> ~ 3 nodes in backend cluster to handle POP3 and SMTP requests
> ~ data stored in SAN, block size 128K, read / write cache enabled
>
> ~ all the nodes have direct access to SAN through fibre channel
>
> ~ In the migration process , when there were 3 nodes , all the systems
> ~ were running smoothly
> ~ but !
> ~ when I added some more load (mailboxes) with 2 more backend nodes, after
> ~ some time 2-3 nodes (any of the nodes in cluster) hanged with the
> ~ following error:
>
> ~ Oct 24 11:04:29 c5 kernel: GFS: Assertion failed on line 599 of file
> ~ inode.c
> ~ Oct 24 11:04:29 c5 kernel: GFS: assertion: "(tmp_gh->gh_flags
> ~ &GL_LOCAL_EXCL) || !(gh->gh_flags & GL_LOCAL_EXCL)"
> ~ Oct 24 11:04:29 c5 kernel: GFS: time = 1095145469 Oct 24 11:04:29 c5
> ~ kernel: GFS: fsid=310farm:gfs01.2: glock = (4, 0)
> ~ Oct 24 11:04:29 c5 kernel:
> ~ Oct 24 11:04:29 c5 kernel: Kernel panic: GFS: Record message above and
> ~ reboot.
>
> ~ (LOCK MANAGER STILL KEEPS RUNNING and other nodes function properly)
> ~ I tried with different configurations of Lock manager but failed to get
> ~ it right.
>
> ~ Is the problem with
>
> ~ GFS (any limitations ?????????)
> ~ I/O
> ~ block size
> ~ lock management
> ~ or (my architecture itself)
>
>
> ~ How can i resolve this issue?
> ~ I am unable to find out proper reason of this.
> ~ Can someone of you who's had hands on this help me out?
>
> ~ Thanks for all those wonderful (non-witty) answers !
> ~ Thorough solutions on the above win a linux friend  :-) !
>
> - - --
> Regards
> Ritesh Agrawal
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.2 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
>
> iD8DBQFBglSEFoz+P95jnTIRAuFsAJ9zK0zu8O6F5tGqZ6I1NSTGnLInzgCbBVU9
> 5qzDkbXxS/DSRGF6ABGYm0Y=
> =gFBU
> -----END PGP SIGNATURE-----