[Libguestfs] Virt-v2v split performance

[Richard W.M. Jones]

[Some random mumblings that I want to put on record ...]

The new "modular virt-v2v" (which I'll probably release as "virt-v2v
2.0" even though it's not really a rewrite) is in the state where it
can convert guests:

Alpha-quality code:
https://github.com/rwmjones/virt-v2v/commits/2021-virt-v2v-split
Original plan:
https://listman.redhat.com/archives/libguestfs/2020-November/msg00022.html

One thing of note is that it's slower in some places and maybe will be
faster in others.  Because I'm using NBD disk pipelines for
everything, when we want to read or write to a local disk image we go
through nbdkit (instead of just accessing the local file as virt-v2v
would do).  However the local file case is not very common in real
world usage (except for testing).

On the flip side, nbdcopy is faster than qemu-img convert when copying
between NBD source and target.  In fact when copying from file to file
through 2 x nbdkits, nbdcopy is about the same speed as qemu-img convert
copying between local files.

Some actual timings.  As a baseline here's original virt-v2v doing a
file to file conversion:

$ virt-v2v -i disk /var/tmp/rhel-8.4.img -o disk -os /var/tmp 
[   0.0] Opening the source -i disk /var/tmp/rhel-8.4.img
[   0.0] Creating an overlay to protect the source from being modified
[   0.1] Opening the overlay
[   4.8] Inspecting the overlay
[   9.9] Checking for sufficient free disk space in the guest
[   9.9] Estimating space required on target for each disk
[   9.9] Converting Red Hat Enterprise Linux 8.4 (Ootpa) to run on KVM
virt-v2v: warning: /files/boot/grub2/device.map/hd0 references unknown 
device "vda".  You may have to fix this entry manually after conversion.
virt-v2v: This guest has virtio drivers installed.
[  48.8] Mapping filesystem data to avoid copying unused and blank areas
[  50.7] Closing the overlay
[  51.2] Assigning disks to buses
[  51.2] Checking if the guest needs BIOS or UEFI to boot
[  51.2] Initializing the target -o local -os /var/tmp
[  51.2] Copying disk 1/1 to /var/tmp/rhel-8.4-sda (raw)
    (100.00/100%)
[  58.3] Creating output metadata
[  58.3] Finishing off


Now virt-v2v 2.0 doing just the conversion step:

+ ./convert/helper-v2v-convert /tmp/v2v
[   0.0] Opening the source
[   4.8] Inspecting the source
[  11.6] Checking for sufficient free disk space in the guest
[  11.6] Converting Red Hat Enterprise Linux 8.4 (Ootpa) to run on KVM
helper-v2v-convert: warning: /files/boot/grub2/device.map/hd0 references 
unknown device "vda".  You may have to fix this entry manually after 
conversion.
helper-v2v-convert: This guest has virtio drivers installed.
[  50.7] Mapping filesystem data to avoid copying unused and blank areas
[  53.9] Closing the overlay
[  54.5] Assigning disks to buses
[  54.5] Checking if the guest needs BIOS or UEFI to boot
[  54.5] Creating output metadata
[  54.5] Finishing off

You can see that it's about 2-3 seconds slower overall.

You have to add to this:

 * About 2 seconds taken to start the input and output nbdkit
   instances.  That's because we start the processes and then wait for
   them to write PID files, in a loop which sleeps for 1 second each
   iteration.  We could be more intelligent here to reduce the sleep
   time, and we could also start the two sides in parallel.

 * nbdcopy takes 7.1 seconds, which is precisely the same time that
   qemu-img convert took (which is quite remarkable considering it's
   doing more work).  We also hope to make nbdcopy much faster.

So you can see what filters and options I'm using, the two nbdkit
commands are:

3008833 pts/3    S+     0:00 nbdkit --exit-with-parent --foreground --newstyle --pidfile /run/user/1000/v2vnbdkit.Fzy4p3/nbdkit1.pid --unix /tmp/v2v/in0 --threads 16 -D nbdkit.backend.datapath=0 --filter cow file file=/var/tmp/rhel-8.4.img fadvise=sequential cache=none
3008863 pts/3    S+     0:00 nbdkit --exit-with-parent --foreground --newstyle --pidfile /run/user/1000/v2vnbdkit.pXctgg/nbdkit1.pid --unix /tmp/v2v/out0 --threads 16 -D nbdkit.backend.datapath=0 file file=/var/tmp/rhel-8.4-sda fadvise=sequential cache=none

I'm not yet using nbdkit-cache-filter.  Old virt-v2v uses copy-on-read
caching.  I'm also not yet using readahead, because nbdkit-readahead-filter
is broken and needs a rewrite.  Old virt-v2v uses readahead and
adjusts it dynamically.

The main aim of the virt-v2v split is not necessarily to make single
instances faster (although we hope because of optimizations in nbdcopy
to do that), but to make the overall system more flexible and take a
holistic overview of what's happening on the system.

The shell script I'm using to automate the new v2v is attached
(because I haven't yet converted the command line parsing code).

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
-------------- next part --------------
A non-text attachment was scrubbed...
Name: V2V.SH
Type: application/x-sh
Size: 1246 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/libguestfs/attachments/20210629/c4daa0ff/attachment.sh>