[Linux-cluster] mod_log_sql killing GFS mount?

[isplist at logicore.net]

Some new information about this.
 From the console;

Nov 25 16:43:11 compdev kernel: GFS: fsid=vgcomp:web.3: Found quota changes 
for 0 IDs
Nov 25 16:43:11 compdev kernel: GFS: fsid=vgcomp:web.3: Done
Nov 25 16:43:18 compdev kernel: GFS: fsid=vgcomp:web.3: fatal: filesystem 
consistency error
Nov 25 16:43:18 compdev kernel: GFS: fsid=vgcomp:web.3:   RG = 31104599
Nov 25 16:43:18 compdev kernel: GFS: fsid=vgcomp:web.3:   function = 
gfs_setbit
Nov 25 16:43:18 compdev kernel: GFS: fsid=vgcomp:web.3:   file = 
/home/xos/gen/updates-2007-10/xlrpm922/rpm/BUILD/gfs-kernel-2.6.9-72/up/src/gf
s/bits.c, line = 71
Nov 25 16:43:18 compdev kernel: GFS: fsid=vgcomp:web.3:   time = 1196030598
Nov 25 16:43:18 compdev kernel: GFS: fsid=vgcomp:web.3: about to withdraw from 
the cluster
Nov 25 16:43:18 compdev kernel: GFS: fsid=vgcomp:web.3: waiting for 
outstanding I/O
Nov 25 16:43:18 compdev kernel: GFS: fsid=vgcomp:web.3: telling LM to withdraw
Nov 25 16:43:20 compdev kernel: lock_dlm: withdraw abandoned memory
Nov 25 16:43:20 compdev kernel: GFS: fsid=vgcomp:web.3: withdrawn



> From the console;

>Nov 25 14:27:11 compdev kernel: GFS: Trying to join cluster "lock_dlm", 
>"vgcomp:web"
>Nov 25 14:27:14 compdev kernel: GFS: fsid=vgcomp:web.3: Joined cluster. Now 
>mounting FS...
>Nov 25 14:27:14 compdev kernel: GFS: fsid=vgcomp:web.3: jid=3: Trying to 
>acquire journal lock...
>Nov 25 14:27:14 compdev kernel: GFS: fsid=vgcomp:web.3: jid=3: Looking at 
>journal...
>Nov 25 14:27:14 compdev kernel: GFS: fsid=vgcomp:web.3: jid=3: Done
>Nov 25 14:27:14 compdev kernel: GFS: fsid=vgcomp:web.3: Scanning for log 
>elements...
>Nov 25 14:27:14 compdev kernel: GFS: fsid=vgcomp:web.3: Found 1 unlinked 
>inodes
>Nov 25 14:27:14 compdev kernel: GFS: fsid=vgcomp:web.3: Found quota changes 
>for 0 IDs
>Nov 25 14:27:14 compdev kernel: GFS: fsid=vgcomp:web.3: Done

>Within seconds of mounting the /Vol_web storage, I get an I/O error;

># df
>Filesystem           1K-blocks      Used Available Use% Mounted on
>/dev/hda1             37689468   2665908  33109016   8% /
>none                    257784         0    257784   0% /dev/shm
>/dev/mapper/VolGroup02-img
>                     505788640      5552 505783088   1% /var/images
>df: `/Vol_web': Input/output error

>I recently installed mod_log_sql on our web servers for central logging. 
>What's strange is that since I've done this, one of my nodes keeps losing one
>of it's GFS mounts. Web servers have the exact same mount yet aren't losing 
>their mount.

>There is no information in /var/log/ messages or any other log file showing 
>what might be going on.

>Any thoughts on what I might be looking for?

>Mike