[dm-devel] [PATCH v2] dm-thin: wakeup worker only when deferred bios exist

[Mike Snitzer]

On Sun, Nov 17 2019 at  8:50pm -0500,
Jeffle Xu <jefflexu at linux.alibaba.com> wrote:

> Single thread fio test (read, bs=4k, ioengine=libaio, iodepth=128,
> numjobs=1) over dm-thin device has poor performance versus bare nvme
> disk.
> 
> Further investigation with perf indicates that queue_work_on() consumes
> over 20% CPU time when doing IO over dm-thin device. The call stack is
> as follows.
> 
> - 40.57% thin_map
>     + 22.07% queue_work_on
>     + 9.95% dm_thin_find_block
>     + 2.80% cell_defer_no_holder
>       1.91% inc_all_io_entry.isra.33.part.34
>     + 1.78% bio_detain.isra.35
> 
> In cell_defer_no_holder(), wakeup_worker() is always called, no matter
> whether the tc->deferred_bio_list list is empty or not. In single thread
> IO model, this list is most likely empty. So skip waking up worker thread
> if tc->deferred_bio_list list is empty.
> 
> A significant IO performance of single thread can be seen with this patch.
> The original IO performance is 448 MiB/s with the fio test previously
> described, while it is 646 MiB/s after applying the patch, which is a
> 44% performance improvement.
> 
> Signed-off-by: Jeffle Xu <jefflexu at linux.alibaba.com>
> ---
> Changes since V1:
> 1. Check whether tc->deferred_bio_list list is empty or not, rather than
> cell->bios as before.
> 2. Retest the performance with the new version of this patch, and update
> the statistics in the commit message.

I've staged this with slight revisions to the header and renamed the
empty variable to has_work, please see:
https://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git/commit/?h=dm-5.5&id=d256d796279de0bdc227ff4daef565aa7e80c898

Thanks.