[dm-devel] should blk-mq halt requeue processing while queue is frozen?

[Bart Van Assche]

On 09/02/2016 09:10 AM, Mike Snitzer wrote:
> On Fri, Sep 02 2016 at 11:12am -0400,
> Mike Snitzer <snitzer at redhat.com> wrote:
>
>> So in the case of blk-mq request-based DM: we cannot expect
>> blk_mq_freeze_queue(), during suspend, to complete if requests are
>> getting requeued to the blk-mq queue via BLK_MQ_RQ_QUEUE_BUSY.
>
> Looking closer at blk-mq.  Currently __blk_mq_run_hw_queue() will move
> any requeued requests to the hctx->dispatch list and then performs async
> blk_mq_run_hw_queue().
>
> To do what you hoped (have blk_mq_freeze_queue() discontinue all use of
> blk-mq hw queues during DM suspend) I think we'd need blk-mq to:
> 1) avoid processing requeued IO if blk_mq_freeze_queue() was used to
>    freeze the queue.  Meaning it'd have to hold requeued work longer
>    than it currently does.
> 2) Then once blk_mq_unfreeze_queue() is called it'd allow requeues to
>    proceed.
>
> This would be catering to a very specific requirement of DM (given it
> re-queues IO back to the request_queue during suspend).
>
> BUT all said, relative to request-based DM multipath, what we have is
> perfectly fine on a correctness level: the requests are re-queued
> because the blk-mq DM device is suspended.
>
> Unfortunately on an efficiency level DM suspend creates a lot of busy
> looping in blk-mq, with 100% cpu usage in a threads with names
> "kworker/3:1H", ideally we'd avoid that!

Hello Mike,

What blk_mq_freeze_queue() does is to wait until queue_rq() has finished 
*and* all pending requests have completed. However, I think in 
dm_stop_queue() all we need is to wait until queue_rq() has finished. 
How about adding new functions in the block layer core to realize this, 
e.g. something like in the attached (untested) patch? Busy looping 
should be avoided - see also the tests of the new "quiescing" flag.

Thanks,

Bart.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-blk-mq-Introduce-blk_mq_quiesce_queue.patch
Type: text/x-patch
Size: 4322 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/dm-devel/attachments/20160902/1238fd75/attachment.bin>