[Virtio-fs] [RFC] [PATCH] virtiofsd: Auto switch between inline and thread-pool processing
Vivek Goyal
vgoyal at redhat.com
Mon Apr 26 17:34:42 UTC 2021
On Fri, Apr 23, 2021 at 05:11:30PM -0400, Vivek Goyal wrote:
> This is just an RFC patch for now. I am still running performance numbers
> and see if this method of switching is good enough or not. I did one run
> and seemed to get higher performance on deeper queue depths. There were
> few tests where I did not match lower queue depth performance with
> no thread pool. May be run to run variation.
Hi All,
So I ran fio jobs with auto switch enabled and uploaded all the data
here. Also pasted final results at the end of email.
https://github.com/rhvgoyal/virtiofs-tests/tree/master/performance-results/auto-switch-inline-thread-pool/april-26-2021
Looks like auto switch threshold of 3 is working pretty well for me. That
is if queue depth of 3 or less, do inline processing otherwise switch
to thread pool. Of course this will vary from system to system and
based on I/O size as well. I have only tested with bs=4K.
I think it might be worth to provide user a knob say
"--thread-pool-threshold=<nr>" where user can control it. By default it
can be 3, may be.
Thanks
Vivek
NAME WORKLOAD Bandwidth IOPS
tpool-16 seqread-psync 62.1mb 15.5k
no-tpool seqread-psync 89.3mb 22.3k
auto-switch-3 seqread-psync 89.6mb 22.4k
auto-switch-4 seqread-psync 88.8mb 22.2k
tpool-16 seqread-psync-multi 256.1mb 64.0k
no-tpool seqread-psync-multi 280.6mb 70.1k
auto-switch-3 seqread-psync-multi 282.9mb 70.7k
auto-switch-4 seqread-psync-multi 284.9mb 71.2k
tpool-16 seqread-libaio 327.7mb 81.9k
no-tpool seqread-libaio 306.5mb 76.6k
auto-switch-3 seqread-libaio 361.8mb 90.4k
auto-switch-4 seqread-libaio 365.3mb 91.3k
tpool-16 seqread-libaio-multi 354.5mb 88.6k
no-tpool seqread-libaio-multi 300.7mb 75.1k
auto-switch-3 seqread-libaio-multi 375.4mb 93.8k
auto-switch-4 seqread-libaio-multi 374.3mb 93.5k
tpool-16 randread-psync 64.5mb 16.1k
no-tpool randread-psync 88.5mb 22.1k
auto-switch-3 randread-psync 87.5mb 21.8k
auto-switch-4 randread-psync 87.7mb 21.9k
tpool-16 randread-psync-multi 268.0mb 67.0k
no-tpool randread-psync-multi 282.5mb 70.6k
auto-switch-3 randread-psync-multi 283.3mb 70.8k
auto-switch-4 randread-psync-multi 282.6mb 70.6k
tpool-16 randread-libaio 325.1mb 81.2k
no-tpool randread-libaio 303.0mb 75.7k
auto-switch-3 randread-libaio 360.8mb 90.2k
auto-switch-4 randread-libaio 361.0mb 90.2k
tpool-16 randread-libaio-multi 352.2mb 88.0k
no-tpool randread-libaio-multi 289.6mb 72.4k
auto-switch-3 randread-libaio-multi 373.5mb 93.3k
auto-switch-4 randread-libaio-multi 372.0mb 93.0k
tpool-16 seqwrite-psync 65.7mb 16.4k
no-tpool seqwrite-psync 87.6mb 21.9k
auto-switch-3 seqwrite-psync 86.5mb 21.6k
auto-switch-4 seqwrite-psync 85.9mb 21.4k
tpool-16 seqwrite-psync-multi 270.9mb 67.7k
no-tpool seqwrite-psync-multi 268.0mb 67.0k
auto-switch-3 seqwrite-psync-multi 267.7mb 66.9k
auto-switch-4 seqwrite-psync-multi 263.7mb 65.9k
tpool-16 seqwrite-libaio 293.5mb 73.3k
no-tpool seqwrite-libaio 286.1mb 71.5k
auto-switch-3 seqwrite-libaio 333.7mb 83.4k
auto-switch-4 seqwrite-libaio 330.5mb 82.6k
tpool-16 seqwrite-libaio-multi 415.0mb 103.7k
no-tpool seqwrite-libaio-multi 281.6mb 70.4k
auto-switch-3 seqwrite-libaio-multi 417.4mb 104.3k
auto-switch-4 seqwrite-libaio-multi 401.6mb 100.4k
tpool-16 randwrite-psync 65.1mb 16.2k
no-tpool randwrite-psync 87.9mb 21.9k
auto-switch-3 randwrite-psync 86.6mb 21.6k
auto-switch-4 randwrite-psync 85.9mb 21.4k
tpool-16 randwrite-psync-multi 268.9mb 67.2k
no-tpool randwrite-psync-multi 261.0mb 65.2k
auto-switch-3 randwrite-psync-multi 261.2mb 65.3k
auto-switch-4 randwrite-psync-multi 263.7mb 65.9k
tpool-16 randwrite-libaio 300.8mb 75.2k
no-tpool randwrite-libaio 287.4mb 71.8k
auto-switch-3 randwrite-libaio 328.2mb 82.0k
auto-switch-4 randwrite-libaio 335.4mb 83.8k
tpool-16 randwrite-libaio-multi 415.4mb 103.8k
no-tpool randwrite-libaio-multi 271.7mb 67.9k
auto-switch-3 randwrite-libaio-multi 415.7mb 103.9k
auto-switch-4 randwrite-libaio-multi 396.5mb 99.1k
tpool-16 randrw-psync 35.3mb/11.7mb 9044/3019
no-tpool randrw-psync 52.9mb/17.7mb 13.2k/4536
auto-switch-3 randrw-psync 52.7mb/17.6mb 13.1k/4516
auto-switch-4 randrw-psync 52.7mb/17.6mb 13.1k/4514
tpool-16 randrw-psync-multi 151.9mb/50.8mb 37.9k/12.7k
no-tpool randrw-psync-multi 160.5mb/53.7mb 40.1k/13.4k
auto-switch-3 randrw-psync-multi 167.6mb/56.1mb 41.9k/14.0k
auto-switch-4 randrw-psync-multi 164.6mb/55.1mb 41.1k/13.7k
tpool-16 randrw-libaio 83.0mb/27.8mb 20.7k/7121
no-tpool randrw-libaio 87.2mb/29.2mb 21.7k/7479
auto-switch-3 randrw-libaio 87.9mb/29.4mb 21.9k/7544
auto-switch-4 randrw-libaio 86.4mb/28.9mb 21.6k/7417
tpool-16 randrw-libaio-multi 201.6mb/67.5mb 50.4k/16.8k
no-tpool randrw-libaio-multi 187.5mb/62.8mb 46.8k/15.7k
auto-switch-3 randrw-libaio-multi 218.0mb/73.0mb 54.5k/18.2k
auto-switch-4 randrw-libaio-multi 213.4mb/71.5mb 53.3k/17.8k
More information about the Virtio-fs
mailing list