[Cluster-devel] [gfs2:gfs2-iopen 12/12] fs/gfs2/util.c:126:3: error: implicit declaration of function 'gfs2_glock_dq_wait'; did you mean 'gfs2_glock_nq_init'?

[Bob Peterson]

----- Original Message -----
> Bob,
> 
> commit "gfs2: Force withdraw to replay journals and wait for it to
> finish" adds three new users of gfs2_glock_dq_wait in function
> signal_our_withdraw. Is the waiting there done for a reason, or can we
> replace gfs2_glock_dq_wait with gfs2_glock_dq / gfs2_glock_dq_uninit
> in that function?
> 
> Thanks,
> Andreas

Hi Andreas,

I remember debugging cases in which we needed to wait.
When we didn't wait for glocks to be demoted, the node just reacquired
the glocks again too quickly, before other nodes could attempt recovery.
When the withdrawing node tried to reacquire them, DLM simply granted
them on the same node, which is the only node that must not do recovery.

Addressing each instance separately:

(1) We would dequeue our journal glock, then dequeue the live glock.
    The other nodes would see the callback for the "live" glock and
    attempt recovery, however they were denied access to the journal
    glock because it was still held on the recovering node. That's
    because the glock state machine (but more importantly the dlm)
    had not yet demoted the lock completely when this took place.
    So none of the nodes performed recovery.
(2) We might be able to get rid of the "wait" for the "live" glock.
    I can't think of a case in which that would behave badly, but
    bear in mind it's been more than a year since I originally wrote
    that. It's probably closer to 2 years now.
(3) We might be able to get rid of the third "wait" which is also for
    the "live" glock. I don't remember the circumstances.

TBH, I wouldn't feel comfortable getting rid of any of those waits
until I do some heavy experimentation on my 5-node cluster with the
recovery tests.

Bob