[Libguestfs] More parallelism in VDDK driver
Eric Blake
eblake at redhat.com
Wed Aug 5 14:49:07 UTC 2020
On 8/5/20 9:38 AM, Eric Blake wrote:
> Correct. Using -W doesn't make sense without -m (if you only have one
> worker, you might as well proceed linearly than trying to randomize
> access, but even when you have multiple threads, there are cases where
> linear operations are still useful, such as 'nbdkit streaming'. But -m
> is definitely the knob that controls how many outstanding I/O requests
> qemu-img is willing to use; and once you are using -m, using -W makes
> life easier for those coroutines to stay active. The default -m1 says
> that at most one request is outstanding, so parallelism in the server is
> not utilized. With higher -m, qemu-img issues up to that many requests
> without waiting for server answers, but all on the same NBD connection.
> Ideally, you'll get the maximum behavior as 'qemu-img -m' and 'nbdkit
> --threads' choose the same values; if either side has fewer in-flight
> operations permitted than the other, then that side has the potential to
> become a bottleneck. Right now, nbdkit defaults to 16 threads (that is,
> up to 16 in-flight operations) for any PARALLEL plugin.
This reminds me of a previous thread: we probably want to add a new mode
to nbdkit parallelism:
https://www.redhat.com/archives/libguestfs/2019-May/msg00089.html
Use of qemu-nbd -m without -W is like my proposed
NBDKIT_THREAD_MODEL_SERIALIZE_RETIREMENT (okay, I just invented that
name; I did not name it in that thread). Use of qemu-nbd -m with -W is
then NBDKIT_THREAD_MODEL_PARALLEL. Use of qemu-nbd without -m (at which
point -W makes no difference) is like
NBDKIT_THREAD_MODEL_SERIALIZE_REQUESTS. The middle mode is where you
allow multiple in-flight requests, but serialize their retirement
(different than serializing their issuance), for a client that can't
deal with out-of-order replies.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
More information about the Libguestfs
mailing list