[Libguestfs] [PATCH 5/6] v2v: Use unordered writes in qemu-img convert

[Nir Soffer]

On Sat, Jan 23, 2021 at 12:45 AM Nir Soffer <nirsof at gmail.com> wrote:
>
> Without unordered writes, qemu-img read data in parallel, but issue
> requests in order, keeping only one in-flight request. With unordered
> writes, qemu keeps up to 8 in-flight requests by default.
>
> With this change we see significant improvement in the
> rhv-upload-plugin. Connection time decreased from 7.08 seconds to 3.77
> seconds, 88% faster. Compared with last version using single connection,
> we are now 42% faster.
>
> [connection 1 ops, 3.778596 s]
> [dispatch 547 ops, 2.977509 s]
> [write 464 ops, 2.703108 s, 355.62 MiB, 131.56 MiB/s]
> [zero 81 ops, 0.118312 s, 1.22 GiB, 10.31 GiB/s]
> [flush 2 ops, 0.000222 s]
>
> [connection 1 ops, 3.774985 s]
> [dispatch 555 ops, 2.976595 s]
> [write 490 ops, 2.685608 s, 352.19 MiB, 131.14 MiB/s]
> [zero 63 ops, 0.122802 s, 1.20 GiB, 9.80 GiB/s]
> [flush 2 ops, 0.000203 s]
>
> [connection 1 ops, 3.777071 s]
> [dispatch 564 ops, 2.930730 s]
> [write 499 ops, 2.631180 s, 323.38 MiB, 122.90 MiB/s]
> [zero 63 ops, 0.119163 s, 1.17 GiB, 9.86 GiB/s]
> [flush 2 ops, 0.000255 s]
>
> [connection 1 ops, 3.778360 s]
> [dispatch 528 ops, 2.979062 s]
> [write 456 ops, 2.676340 s, 318.00 MiB, 118.82 MiB/s]
> [zero 70 ops, 0.118221 s, 1.08 GiB, 9.18 GiB/s]
> [flush 2 ops, 0.000202 s]
>
> Signed-off-by: Nir Soffer <nsoffer at redhat.com>
> ---
>  v2v/v2v.ml | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/v2v/v2v.ml b/v2v/v2v.ml
> index 54f26297..aa9d9713 100644
> --- a/v2v/v2v.ml
> +++ b/v2v/v2v.ml
> @@ -763,6 +763,7 @@ and copy_targets cmdline targets input output =
>          [ "-n"; "-f"; "qcow2"; "-O"; output#transfer_format t ] @
>          (if cmdline.compressed then [ "-c" ] else []) @
>          [ "-S"; "64k" ] @
> +        [ "-W"; ] @

If output is not using a parallel thread model, this has no benefit
and it may cause fragmentation in the output storage, or even
worse performance.

So maybe:

    (if output#parallel then [ "-W" ] else []) @

Or:

    (if output#threads > 1 then [ "-W" ] else []) @

>          [ overlay_file; filename ] in
>        let start_time = gettimeofday () in
>        if run_command cmd <> 0 then
> --
> 2.26.2
>