[Cluster-devel] "->ls_in_recovery" not released

[David Teigland]

On Mon, Nov 22, 2010 at 05:31:25PM +0100, Menyhart Zoltan wrote:
> We have got a two-node OCFS2 file system controlled by the pacemaker.

Are you using dlm_controld.pcmk?  If so, please try the latest versions of
pacemaker that use the standard dlm_controld.  The problem may be related
to the lockspace membership events that are passed to the kernel from
dlm_controld.  'dlm_tool dump' from each node, correlated with the
corosync membership events, may probably reveal the problem.  Start by
looking at the sequence of confchg log messages,
e.g. "dlm:ls:g conf 3 1 0 memb 1 2 4 join 4 left"

conf
3 = number of members
1 = number of members that joined
0 = number of members that left

"memb 1 2 4" - nodeids of members
"join 4" - nodeids of members that joined
"left" - nodeids of members that left

> "ls_recover()" includes several other cases when it simply goes
> to the "fail:" branch without setting free "->ls_in_recovery" and
> without cleaning up the inconsistent data left behind.
> 
> I think some error handling code is missing in "ls_recover()".
> Have you modified the DLM since the RHEL 6.0?

No, in_recovery is supposed to remain locked until recovery completes.
Any number of ls_recover() calls can fail due to more member changes
during recovery, but one of them should eventually succeed (complete
recovery), once the membership stops changing.  Then in_recovery will be
unlocked.

Look at the specific errors causing ls_recover() to fail, and check if
it's a confchg-related failure (like above), or another kind of error.

Dave