[Libguestfs] [PATCH 2/5] v2v/lib/util.ml: Get disk allocation from input

[Nir Soffer]

On Sun, Dec 19, 2021 at 1:30 AM Nir Soffer <nsoffer at redhat.com> wrote:
>
> On Sun, Dec 19, 2021 at 12:40 AM Nir Soffer <nsoffer at redhat.com> wrote:
> >
> > On Sat, Dec 18, 2021 at 11:33 PM Richard W.M. Jones <rjones at redhat.com> wrote:
> > >
> > > On Sat, Dec 18, 2021 at 10:36:30PM +0200, Nir Soffer wrote:
> > > > After finalizing the transfer, virt-v2v try to connect to the output
> > > > socket and query disk allocation. This may work for some outputs
> > > > supporting block status, but for rhv_upload output this cannot work for
> > > > 2 reasons:
> > > > - The rhv-upload-plugin does not support extents
> > > > - The transfer was finalized before this call, so the plugin lost access
> > > >   to the image.
> > > >
> > > > Here is an example failure log:
> > > >
> > > > [  74.2] Creating output metadata
> > > > python3 '/tmp/v2v.WMq8Tk/rhv-upload-finalize.py' '/tmp/v2v.WMq8Tk/params6.json'
> > > > finalizing transfer b03fe3ba-a4ff-4634-a0a0-10b3daba3cc2
> > > > ...
> > > > transfer b03fe3ba-a4ff-4634-a0a0-10b3daba3cc2 finalized in 2.118 seconds
> > > > ...
> > > > nbdkit: debug: accepted connection
> > > > ...
> > > > nbdkit: python[4]: debug: python: close
> > > > virt-v2v: error: exception: NBD.Error("nbd_block_status: request out of
> > > > bounds: Invalid argument", 22)
> > > >
> > > > Fix by using the input socket.
> > > > ---
> > > >  lib/utils.ml | 2 +-
> > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/lib/utils.ml b/lib/utils.ml
> > > > index d6861d08..f6c85543 100644
> > > > --- a/lib/utils.ml
> > > > +++ b/lib/utils.ml
> > > > @@ -171,21 +171,21 @@ let with_nbd_connect_unix ~socket ~meta_contexts ~f =
> > > >      ~f:(fun () ->
> > > >            List.iter (NBD.add_meta_context nbd) meta_contexts;
> > > >            NBD.connect_unix nbd socket;
> > > >            protect
> > > >              ~f:(fun () -> f nbd)
> > > >              ~finally:(fun () -> NBD.shutdown nbd)
> > > >         )
> > > >      ~finally:(fun () -> NBD.close nbd)
> > > >
> > > >  let get_disk_allocated ~dir ~disknr =
> > > > -  let socket = sprintf "%s/out%d" dir disknr
> > > > +  let socket = sprintf "%s/in%d" dir disknr
> > >
> > > This patch is definitely wrong - we need to get the allocation size
> > > from the output disk.  Options such as -oa preallocated, and just
> > > general issues like block size, nbdcopy sparseness detection etc,
> > > would affect this.
> > >
> > > It probably indicates a problem with rhv-upload-plugin (again) that
> > > it's not really prepared to be an idempotent part of a disk image
> > > pipeline.
> >
> > This patch also definitely works for rhv-upload, we create working
> > vms.
> >
> > It can be fixed in another way, maybe skipping the check when using
> > rhv-upload because it cannot provide the needed info.
> >
> > I'm not sure why this check is needed - RHV already has everything
> > it needs at this point, and it does not care about disk allocation after
> > the disk was uploaded. If RHV needs any info about the disk, it can
> > get it from storage.
> >
> > Maybe you add stuff to the ovf that RHV does need?
>
> I see that this code is not in rhel-9.0.0 branch, so this patch is not
> needed for https://bugzilla.redhat.com/2032324
>
> Laszlo, can you explain why we need the number of allocated bytes
> after the disk was already converted to the target storage?
>
> Looking at the bug:
> https://bugzilla.redhat.com/2027598
>
> This is a fix for the issue in the old rhv output, which is deprecated and
> should not be used in new code, and it is not compatible with rhv-upload
> which is the modern way to import into RHV.
>
> Also this cannot work for RHV now, so we need a way to disable
> this when importing to RHV.
>
> We can expose the block status in RHV plugin,

This patch adds extents to rhv-upload-plugin:
https://listman.redhat.com/archives/libguestfs/2021-December/msg00197.html

It depends on fixing the python plugin extents callback:
https://listman.redhat.com/archives/libguestfs/2021-December/msg00196.html



> but to use this we must
> call block status after the disk is converted, but before the output
> is finalized, since when you finalize RHV disconnect the disk from
> the host, and the RHV plugin cannot access it any more.
>
> The flow for rhv-upload should be:
>
>     run nbdcopy
>     query block status
>     finalize output
>     create ovf
>
> Nir