[Libguestfs] [PATCH v2v] vddk: Use blocksize filter to split very large requests

[Laszlo Ersek]

On 02/15/22 14:43, Richard W.M. Jones wrote:
> We know that VDDK has problems handling very large single requests.
> It may cause out of memory errors on the server unless a relatively
> obscure server-side configuration change is made.  With modular
> virt-v2v we won't necessarily have fine control over who is connecting
> to the input socket and what sized requests they may make, eg if we
> delegate copying to a third party.
> 
> nbdkit-blocksize-filter offers a simple solution.  It will split
> requests larger than a certain size, ensuring that whatever NBD
> requests come in, we shouldn't cause problems on the VMware server.
> I chose 2M as the maximum request size, but this could be fine-tuned
> later.
> 
> For a longer explanation, see:
> https://gitlab.com/nbdkit/nbdkit/-/commit/6f4746af9ac65c779d6afda9459e9f862a26d145
> ---
>  input/nbdkit_vddk.ml | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/input/nbdkit_vddk.ml b/input/nbdkit_vddk.ml
> index 40dae9baa7..9609c4423f 100644
> --- a/input/nbdkit_vddk.ml
> +++ b/input/nbdkit_vddk.ml
> @@ -144,6 +144,16 @@ See also the virt-v2v-input-vmware(1) manual.") libNN
>     *)
>    Nbdkit.add_filter_if_available cmd "cacheextents";
>  
> +  (* Split very large requests to avoid out of memory errors on the
> +   * server.  Since we're using this filter, also add minblock=512
> +   * although it will make no difference.
> +   *)
> +  if Nbdkit.probe_filter "blocksize" then (
> +    Nbdkit.add_filter cmd "blocksize";
> +    Nbdkit.add_arg cmd "minblock" "512";
> +    Nbdkit.add_arg cmd "maxdata" "2M"
> +  );
> +
>    (* IMPORTANT! Add the COW filter.  It must be furthest away
>     * except for the multi-conn and rate filters.
>     *)
> 

Acked-by: Laszlo Ersek <lersek at redhat.com>