[dm-devel] Semaphore bug in dm-thin

[Joe Thornber]

Thanks Mikulas, Good analysis.  Looking at this now.

On Thu, Oct 20, 2011 at 02:59:12PM -0400, Mikulas Patocka wrote:
> Hi Joe
> 
> Zdenek recently got a lockdep warning in dm-thin. See this: 
> http://pastebin.test.redhat.com/65346
> 
> I looked at the warning and realized that it is not a false alarm, but a 
> real bug.
> 
> Lockdep warns about dependency &md->io_lock -> &md->map_lock, but io_lock 
> is a semaphore and map_lock is a spinlock, so such lock ordering should be 
> always allowed (you can't take them in reverse order, because you can't 
> take a semaphore while holding a spinlock).
> 
> Looking deeper in the warning, it complains that there is an exeisting 
> dependency chain &md->map_lock --> &pmd->root_lock --> &md->io_lock
> 
> Here, map_lock is a spinlock and root_lock is a semaphore, this order of 
> taking locks is obviously wrong.
> 
> Cutting relevant part of the backtrace, we see:
> 41. [ 3836.656384] -> #1 (&pmd->root_lock){+++++.}:
> 42. [ 3836.656384]        [<ffffffff81090740>] lock_acquire+0x90/0x1f0
> 43. [ 3836.656384]        [<ffffffff814ac1ce>] down_read+0x3e/0x86
> 44. [ 3836.656384]        [<ffffffffa02aaafe>]
>     dm_pool_get_data_dev_size+0x2e/0x60 [dm_thin_pool]
> 45. [ 3836.656384]        [<ffffffffa02a615b>]
>     thin_iterate_devices+0x3b/0x80 [dm_thin_pool]
> 46. [ 3836.656384]        [<ffffffffa006b2bf>]
>     dm_table_supports_flush+0x5f/0x90 [dm_mod]
> 47. [ 3836.656384]        [<ffffffffa006d2ec>]
>     dm_table_set_restrictions+0x9c/0x180 [dm_mod]
> 48. [ 3836.656384]        [<ffffffffa006a88a>] dm_swap_table+0x1da/0x2e0
>     [dm_mod]
> 49. [ 3836.656384]        [<ffffffffa00701db>] dev_suspend+0x11b/0x260
>     [dm_mod]
> 50. [ 3836.656384]        [<ffffffffa007089a>] ctl_ioctl+0x15a/0x2c0
>     [dm_mod]
> 
> dm_swap_table calls __bind (which is not shown because it is inlined), 
> __bind takes a spinlock &md->map_lock and calls dm_table_set_restrictions, 
> from there, the code goes to thin_iterate_devices and it tries to take a 
> semaphore &pmd->root_lock.
> 
> So there is a clear bug, dm-thin is taking a semaphore while spinlock 
> &md->map_lock is held.
> 
> Mikulas