[Cluster-devel] [GFS2 PATCH] GFS2: Defer the final inode glock put

[Bob Peterson]

----- Original Message -----
> Hi,
> 
> On 11 February, Steve Whitehouse shot a hole through my latest attempt
> to solve the livelock related to GFS2 and the inode shrinker.
> The following is a new attempt, and it seems to be working well in
> testing.
> 
> This patch approaches the problem as a purely glock-related problem.
> The problem is that the final glock put can't be done by inode evict
> (or it results in the livelock as described below). This solution
> simply defers the "eviction put" so that it is done by way of a new
> function in the quota daemon.
> 
> One advantage of this method is that deferring the put shouldn't
> cause any problems if another process comes in and re-uses the glock
> for a new incarnation of the dinode. The existing code will take the
> glock off the lru list, and reuses it. In that case the "eviction put"
> won't be the final put, so it won't tell the DLM to unlock the
> lkb out of order (as was problematic with earlier patches); it will
> just adjust the reference count as needed. Note that this, in theory,
> could happen multiple times in a row, so we keep a counter rather
> than a flag. This is similar to how GFS1 handled this situation.
> ---
> Patch description:
> 
> This patch tries to solve the livelock problem that goes like this:
> 1. An inode is being evicted from memory, which calls clear_glock
> 2. clear_glock does a final glock_put which calls into DLM to tell
>    DLM to do the final unlock of the glock.
> 3. DLM calls into comm to do the unlock
> 4. The comm layer needs memory to satisfy the DLM communication
> 5. The memory allocate blocks due to lack of memory, and so the VFS
>    inode shrinker is called to free memory.
> 6. The inode shrinker calls into GFS2 to free one of its glocks
> 7. Goto 1.
> The patch avoids this circular dependency by making the final glock
> put done for inodes happen later. After the glock is queued to the
> glock LRU, it wakes up the quota daemon. A new function of the
> quota daemon checks if there are any inode glocks on the LRU
> list that have a deferred glock_put, and if so, does the put.
> This allows the inode shrinker to proceed, since the final put
> is done later. The final put, called from the quota daemon, has no
> dependencies, and makes the call into DLM to do the final unlock.
> 
> Signed-off-by: Bob Peterson <rpeterso at redhat.com>
> ---

Hi,

Self-NACK for now.
In testing I discovered a rare race condition that seems to have
caused lru list corruption. I'm investigating now.

Regards,

Bob Peterson
Red Hat File Systems