[dm-devel] [PATCH v2] dm-bufio: fix deadlock with loop device

[Junxiao Bi]

When thin-volume was built on loop device, if available memory is low,
the following deadlock can be triggered.

One process P1 was allocating memory with GFP_FS flag, direct alloc fail,
memory reclaim invoked memory shrinker in dm_bufio, dm_bufio_shrink_scan()
run, mutex dm_bufio_client->lock was acquired, then P1 wait for dm_buffer
io done in __try_evict_buffer->()__try_evict_buffer().

But this io may never done as it was issued to the underlying loop device
who forward it using fs direct-io, there some memory allocation were using
GFP_FS(like do_blockdev_direct_IO()), if direct alloc fail, memory reclaim
will invoke memory shrinker in dm_bufio, where dm_bufio_shrink_scan()
will be invoked, since the mutex was already hold by P1, the loop thread
will hung, io will never done. ABBA deadlock.

Signed-off-by: Junxiao Bi <junxiao.bi at oracle.com>
---
Changes in v2:
  - refine the commit log

 drivers/md/dm-bufio.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/drivers/md/dm-bufio.c b/drivers/md/dm-bufio.c
index 2a48ea3f1b30..b6b5acc92ca2 100644
--- a/drivers/md/dm-bufio.c
+++ b/drivers/md/dm-bufio.c
@@ -1599,9 +1599,7 @@ dm_bufio_shrink_scan(struct shrinker *shrink, struct shrink_control *sc)
 	unsigned long freed;
 
 	c = container_of(shrink, struct dm_bufio_client, shrinker);
-	if (sc->gfp_mask & __GFP_FS)
-		dm_bufio_lock(c);
-	else if (!dm_bufio_trylock(c))
+	if (!dm_bufio_trylock(c))
 		return SHRINK_STOP;
 
 	freed  = __scan(c, sc->nr_to_scan, sc->gfp_mask);
-- 
2.17.1