<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8">
<META NAME="GENERATOR" CONTENT="GtkHTML/3.0.10">
</HEAD>
<BODY>
Looking at the logs, this problem started at 16:04 yesterday. This set of log messages has been logged every 10 minutes since then.<BR>
<BR>
Any ideas?<BR>
<BR>
Matt<BR>
<BR>
On Mon, 2005-12-12 at 10:35, Matt Brookover wrote:
<BLOCKQUOTE TYPE=CITE>
<FONT COLOR="#737373"><I>We are getting processes stuck in device waits on one file system. These errors are logged in /var/log/messages:<BR>
<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: stuck in gfs_releasepage()...<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: blkno = 12446334, bh->b_count = 9<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: bh->b_journal_head = !NULL<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: gl = (4, 12477424)<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: bd_new_le.le_trans = NULL<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: bd_incore_le.le_trans = NULL<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: bd_frozen = NULL<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: bd_pinned = 0<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: bd_ail_tr = NULL<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip = 12477424/12477424<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_count = 1, ip->i_vnode = !NULL<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[0] = NULL<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[1] = NULL<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[2] = NULL<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[3] = NULL<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[4] = NULL<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[5] = NULL<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[6] = NULL<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[7] = NULL<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[8] = NULL<BR>
Dec 12 10:04:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[9] = NULL<BR>
Dec 12 10:09:17 imagine su(pam_unix)[5104]: session closed for user root<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: stuck in gfs_releasepage()...<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: blkno = 12446334, bh->b_count = 9<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: bh->b_journal_head = !NULL<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: gl = (4, 12477424)<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: bd_new_le.le_trans = NULL<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: bd_incore_le.le_trans = NULL<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: bd_frozen = NULL<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: bd_pinned = 0<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: bd_ail_tr = NULL<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip = 12477424/12477424<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_count = 1, ip->i_vnode = !NULL<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[0] = NULL<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[1] = NULL<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[2] = NULL<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[3] = NULL<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[4] = NULL<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[5] = NULL<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[6] = NULL<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[7] = NULL<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[8] = NULL<BR>
Dec 12 10:14:02 imagine kernel: GFS: fsid=CSM_ACN:admin01.5: ip->i_arch.i_cache[9] = NULL<BR>
<BR>
The file system in question appears to work fine on the other nodes, I unmounted it to be on the safe side.<BR>
<BR>
This is redhat enterprise 3.6, kernel 2.4.21-37.ELsmp, GFS 6.0.2.27-0. GFS was built from the source.<BR>
There are 2 partitions in the admin pool, the second was added a week or so ago.<BR>
<BR>
I tried to unmount it, but the umount failed because of the processes that are stuck in device waits.<BR>
<BR>
Any ideas?<BR>
<BR>
thank you<BR>
<BR>
Matt</FONT><BR>
<A HREF="mailto:mbrookov@mines.edu"><U>mbrookov@mines.edu</U></A><BR>
<FONT COLOR="#737373"><BR>
<BR>
<BR>
<HR>
<PRE>--
Linux-cluster mailing list
Linux-cluster@redhat.com</FONT>
<A HREF="https://www.redhat.com/mailman/listinfo/linux-cluster"><U>https://www.redhat.com/mailman/listinfo/linux-cluster</U></I></A></PRE>
</BLOCKQUOTE>
</BODY>
</HTML>