[Libguestfs] [PATCH v7 6/6] v2v: Add -o rhv-upload output mode (RHBZ#1557273).

[Richard W.M. Jones]

On Sun, Mar 25, 2018 at 08:05:14PM +0000, Nir Soffer wrote:
> On Sun, Mar 25, 2018 at 2:41 PM Richard W.M. Jones <rjones at redhat.com>
> wrote:
> > >         initial_size=image_size,
> > >         provisioned_size=image_info["virtual-size"],
> >
> > Can you describe exactly what these two sizes mean?
> 
> - provisioned_size is the virtual size of the image.
> - initial_size is used only for thin provisioned disk on block storage.
> This is
> the size of the logical volume we created for this disk.

If initial_size is too small what happens?  Does it get
extended on the fly as we upload data?  I'm guessing no because ...

> On thin disk on block storage you will not be able to write or zero more
> then initial_size bytes.

So it sounds as if we'll need to set both to the virtual size.

> When a vm is running, vdsm monitor the allocated space and ask the SPM
> host to allocated more space when the highest write offset is too high, so
> the disk looks like a thin provisioned disk to the vm. This mechanism is
> not available for storage operations, and all of them specify initial_size
> when converting images to qcow2 format.

This is going to cause problems for backup applications too.  There's
no way in general they can estimate the data to be transferred.

> > Remember that virt-v2v works by streaming.  At the point where we are
> > calling this API virt-v2v only knows the virtual size.  We never know
> > the size of the final qcow2 file.
> >
> 
> If you don't have a way to estimate the final size you need to allocated
> the entire image when you create a disk.
> 
> But I don't understand why you don't have access to the image. I understood
> that the flow is:
> 
> image file -> qemu-img convert -> nbdkit -> ovirt-imageio -> ovirt disk

The source is usually remote, and is often not a real file (eg. it's a
virtual file or a live P2V transfer or something like that).

We do indeed have an estimate of the image size after fstrimming etc
but I don't think it's very accurate.

> In this flow you can estimate the size using the image file before starting
> the streaming process.
> 
> If there is no way to estimate the size and you must allocate the entire
> image, we can add a shrink step in upload finalization, to shrink the image
> size to optimal size. We are already doing this in several flows that
> cannot estimate the final disk size and do over allocation.

I guess we'll need something like that.

> [snipped]
> 
> > > > +        log = logging.getLogger(),
> >
> > > +        insecure = params['insecure'],
> > > >
> > >
> > > If ca_file cannot be None, then insecure is not needed, based
> > > on Ondra review from earlier version.
> >
> > Is this really true?  My reading of the code is that the insecure flag
> > verifies the server to the client, whereas the certificate bundle is
> > for verifying the client to the server.
> >
> 
> Maybe, I'm not familiar with the SDK.
> 
> What is the motivation for disabling this flag?

Usually because people never bother to set up certificates, or if they
try they fail because it's ridiculously complicated.

Will follow up on the remaining points in a later email.

Thanks,

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org