[Libguestfs] [PATCH v2v] input: Disable multi-conn in VDDK input mode

[Laszlo Ersek]

On 12/06/21 23:05, Richard W.M. Jones wrote:
> The cow filter unconditionally enables multi-conn (because it is
> safe).  However this causes an unintended consequence with the VDDK
> plugin.  Multiple VDDK handles are opened (one per multi-conn
> connection), and for some reason, possibly internal locking, they
> conflict with each other.  This manifests itself as API calls taking
> between 2 and 7 times longer to serve (especially QueryAllocatedBlocks
> which seems to slow down most).
> 
> Avoid this by adding nbdkit-multi-conn-filter with
> multi-conn-mode=disable on top which disables multi-conn
> advertisement.
> 
> Virt-v2v -ic esx://... -it vddk -o null, before this change:
> 
> [   2.8] Opening the source
> [  11.4] Inspecting the source
> [  16.2] Checking for sufficient free disk space in the guest
> [  16.2] Converting Windows 10 Enterprise to run on KVM
> virt-v2v: This guest has virtio drivers installed.
> [  25.5] Mapping filesystem data to avoid copying unused and blank areas
> [  67.3] Closing the overlay
> [  68.0] Assigning disks to buses
> [  68.0] Checking if the guest needs BIOS or UEFI to boot
> [  68.0] Copying disk 1/1
> █ 100% [****************************************]
> [ 416.4] Creating output metadata
> [ 416.4] Finishing off
> 
> After this change:
> 
> [   2.8] Opening the source
> [  12.6] Inspecting the source
> [  18.7] Checking for sufficient free disk space in the guest
> [  18.7] Converting Windows 10 Enterprise to run on KVM
> virt-v2v: This guest has virtio drivers installed.
> [  36.1] Mapping filesystem data to avoid copying unused and blank areas
> [  77.0] Closing the overlay
> [  77.9] Assigning disks to buses
> [  77.9] Checking if the guest needs BIOS or UEFI to boot
> [  77.9] Copying disk 1/1
> █ 100% [****************************************]
> [ 332.3] Creating output metadata
> [ 332.3] Finishing off
> ---
>  input/nbdkit_vddk.ml | 22 +++++++++++++++++++++-
>  1 file changed, 21 insertions(+), 1 deletion(-)
> 
> diff --git a/input/nbdkit_vddk.ml b/input/nbdkit_vddk.ml
> index 79ec363bce..c2c3cceb93 100644
> --- a/input/nbdkit_vddk.ml
> +++ b/input/nbdkit_vddk.ml
> @@ -157,10 +157,30 @@ See also the virt-v2v-input-vmware(1) manual.") libNN
>    let cmd = Nbdkit.add_filter_if_available cmd "cacheextents" in
>  
>    (* IMPORTANT! Add the COW filter.  It must be furthest away
> -   * except for the rate filter.
> +   * except for the multi-conn and rate filters.
>     *)
>    let cmd = Nbdkit.add_filter cmd "cow" in
>  
> +  (* The cow filter unconditionally enables multi-conn (because it is
> +   * safe).  However this causes an unintended consequence with the VDDK
> +   * plugin.  Multiple VDDK handles are opened (one per multi-conn
> +   * connection), and for some reason, possibly internal locking, they
> +   * conflict with each other.  This manifests itself as API calls taking
> +   * between 2 and 7 times longer to serve (especially QueryAllocatedBlocks
> +   * which seems to slow down most).

I wonder if the lock contention is in the vddk library, or on the vmware
server side... QueryAllocatedBlocks seems to be a read-only operation even.

Acked-by: Laszlo Ersek <lersek at redhat.com>

(I know it's already upstream, but still.)

Laszlo

> +   *
> +   * Avoid this by adding nbdkit-multi-conn-filter with
> +   * multi-conn-mode=disable on top which disables multi-conn
> +   * advertisement.
> +   *)
> +  let cmd =
> +    if Nbdkit.probe_filter "multi-conn" then (
> +      let cmd = Nbdkit.add_filter cmd "multi-conn" in
> +      let cmd = Nbdkit.add_arg cmd "multi-conn-mode" "disable" in
> +      cmd
> +    )
> +    else cmd in
> +
>    (* If the filter supports it, enable cow-block-size (added in
>     * nbdkit 1.27.6).  This helps to reduce fragmentated small
>     * extent and read requests.
>