[Linux-cluster] Question about gfs2_tools lockdump

[Scooter Morris]

Hi all,
     I've got a 5 node RHEL 5.5 cluster with a number of gfs2 
filesystems.  After a lot of effort (and help from RedHat) we've gotten 
to the stage where the cluster is quite stable, but now we're starting 
to see some performance degradation.   In investigating this, I've been 
poking around and I'm seeing some things that I can't explain.  In 
particular, on a quite filesystem (no processes according to lsof on all 
nodes), a gfs2_tool lockdump gives 1,000's of lock entries (G: lines).  
Of those several have R: entries (resource group?) and several have H: 
entries.  The H: entries are particularly strange because all H: entries 
are of the form:
     H: s:EX f:H e:0 p:8953 [(ended)] ...

My understanding is that this indicates a lock holder with an exclusive 
lock, but the process has ended (?).   Why aren't these locks going 
away?  Shouldn't they be cleared after the process ends (particularly 
since some of them are exclusive locks...)?  Any help in understanding 
these entries would be very helpful.

-- scooter