[Libguestfs] [PATCH] v2v: Output saved overlays in a machine-readable fashion

[Martin Kletzander]

On Fri, Oct 11, 2019 at 09:37:18AM +0200, Martin Kletzander wrote:
>On Thu, Oct 10, 2019 at 03:33:25PM +0100, Richard W.M. Jones wrote:
>>On Wed, Oct 09, 2019 at 02:19:46PM +0200, Martin Kletzander wrote:
>>> Even though this option is not to be used according to the manual, it:
>>>
>>>  a) still might be useful even for machine-readable logs
>>>
>>>  b) should not break the machine-readable output
>>
>>I'm a bit confused what you're trying to do here.
>>
>
>When working on a conversion where the disk is already copied from the source
>hypervisor to the destination storage (due to various reasons, e.g. having
>faster access to one of the storages or being able to use the disk_sync script
>to prefetch some of the data before the machine gets shut down) I need to
>utilize virt-v2v for two specific purposes:
>
> 1) to create the output metadata and
>
> 2) to make the changes in the guest disks (which are already copied)
>
>but not for the third thing that virt-v2v does as well, which is copying the
>disks from source to destination.
>
>For case #1 I initially used virt-v2v --no-copy, which creates the metadata, but
>discards the changes made to the disks in the overlays and for #2 I used
>--in-place which changes the data on the source (for which I crafted a specific
>XML to be used with -i libvirtxml), but does not output any metadata for the
>destination.
>
>There are couple of issues I am facing with this:
>
> a) I need to run the conversion twice
>
> b) --in-place does not know about the destination (and it does not do the same
>    thing as if it knew)
>
> c) It should only be done using --no-copy first and --in-place second just to
>    make sure the modifications are not done twice (although this is just to
>    stay on the safe side).
>
>It still felt like a massive hack to me and while I still held on to the idea of
>keeping the overlays around without actually copying the data anywhere I found
>the --debug-overlays option.  It fits my use case perfectly because what I am
>doing now (with this patch) is:
>
> i) run virt-v2v --no-copy with the proper destination
>
> ii) take the overlays and run qemu-img commit on them
>
> iii) remove the overlays
>
>This way all of my concerns are solved and the only one that arises is the fact
>that --debug-overlays is documented as "... only used for debugging and may be
>removed in a future version."
>
>One interesting thing is that I only found that information in the man page
>after I wrote this patch and started using it.  The reason for it was that I
>started actually writing something very similar to this and I stumbled upon the
>code that already does that (with --debug-overlays).
>
>I also thought that if someone runs virt-v2v --machine-readable --debug-overlays
>that the machine readable output would be broken because the output is not json,
>but it looks like (I'm just trying it now) --machine-readable does not write any
>data to stdout or stderr, even when specified as
>--machine-readable=stream:stdout (or stderr) for simple libvirt2libvirt
>migration.  So point (b) from the commit message is probably invalid.  Oh, actually, I don't see it being used for anything else than printing apabilities and estimates.  I wonder where the json messages in machine readable logfile that virt-v2v-wrapper requests are coming from.
>
>One last thing is that currently any application that wants to run virt-v2v (virt-v2v-wrapper in my case) needs to parse the output to get any meaningful data from it.  What I tried here was using a new type of message called 'data' that upper applications could just parse and use.  In the long run I could add more data to the output so that it is easier to use from virt-v2v-wrapper and other applications.  But that is not something that is necessary for this particular patch (or the discussion about the --debug-overlays being supported in the long term), it just made it easier.
>
>Does that make sense?
>
>For some more context/info, see How I used it in a PoC here:
>
>  https://github.com/nertpinx/v2v-conversion-host/blob/asdf/wrapper/log_parser.py#L71
>

It's also a way easier and sustainable way of what I implemented in a different
and weird way on top of this ancient patch:

  https://www.redhat.com/archives/libguestfs/2019-April/msg00253.html

where virt-v2v was the one who actually ran qemu-img commit.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/libguestfs/attachments/20191011/35daf334/attachment.sig>