[Libguestfs] [PATCH v2] v2v: rhv-upload-plugin: Defer imageio connection

[Nir Soffer]

On Thu, Jan 21, 2021 at 1:57 PM Nir Soffer <nirsof at gmail.com> wrote:
...

How I tested:

I could not use the example python plugin with the added delay, since
os.preadv() is not available in python 3.6 (added in 3.7).

I could not use the delay filter since it supports only integer delay. With
fedora 32 image with 231 extents, qemu-img calls block_status 960 times
during import and this is too slow to test.

So I used this hack:

$ git diff
diff --git a/plugins/file/file.c b/plugins/file/file.c
index 1651079a..99a95af0 100644
--- a/plugins/file/file.c
+++ b/plugins/file/file.c
@@ -785,6 +785,8 @@ do_extents (void *handle, uint32_t count, uint64_t offset,
   const bool req_one = flags & NBDKIT_FLAG_REQ_ONE;
   uint64_t end = offset + count;

+  usleep(300000);
+
   do {
     off_t pos;

Started nbdkit with fedora 32 image:

$ ./nbdkit -f -v ./plugins/file/.libs/nbdkit-file-plugin.so
file=/var/tmp/fedora-32.raw

Add overlay to make it work with virt-v2v -i disk:

$ qemu-img create -f qcow2 -b nbd://localhost -F raw /var/tmp/fedora-32.qcow2

With this, getting all extents takes more than 60 seconds, enough to get the
connection closed:

$ time qemu-img map --output json nbd://localhost >/dev/null

real 1m9.416s
user 0m0.007s
sys 0m0.017s

I imported the qcow2 image on ovirt host, with --oo rhv_direct=true/false.

Looking at imageio log, we see that the initial connection is closed,
and the second
connection is made once qemu finish to get extetents for the entire image.

I also tried to import using imageio client, using this pipeline:

    imageio nbd client -> qemu-nbd -> overlay.qcow2 -> nbdkit

On imageio side, we always try to get extents for 2g, and we don't use
NBD_CMD_FLAG_REQ_ONE. But on nbdkit we see that all extents
calls use req_one=1.

Looks like the issue is this code in qemu nbd driver:

1676 static int coroutine_fn nbd_client_co_block_status(
1677         BlockDriverState *bs, bool want_zero, int64_t offset,
int64_t bytes,
1678         int64_t *pnum, int64_t *map, BlockDriverState **file)
1679 {
...
1685     NBDRequest request = {
1686         .type = NBD_CMD_BLOCK_STATUS,
1687         .from = offset,
1688         .len = MIN(QEMU_ALIGN_DOWN(INT_MAX, bs->bl.request_alignment),
1689                    MIN(bytes, s->info.size - offset)),
1690         .flags = NBD_CMD_FLAG_REQ_ONE,
1691     };

Also qemu-img convert is getting extents twice:

1. One call per extents before starting the copy:

nbdkit: file.0: debug: file: extents count=196608 offset=851968 req_one=1
nbdkit: file.1: debug: file: extents count=196608 offset=1376256 req_one=1
nbdkit: file.3: debug: file: extents count=393216 offset=1638400 req_one=1
nbdkit: file.4: debug: file: extents count=458752 offset=2686976 req_one=1
nbdkit: file.5: debug: file: extents count=983040 offset=3211264 req_one=1
...
nbdkit: file.14: debug: file: extents count=524288 offset=6441402368 req_one=1

2. One call per extent before each pread(), during copy:

nbdkit: file.0: debug: file: extents count=196608 offset=851968 req_one=1
nbdkit: file.1: debug: file: extents count=196608 offset=1376256 req_one=1
nbdkit: file.3: debug: file: extents count=393216 offset=1638400 req_one=1
nbdkit: file.4: debug: file: extents count=458752 offset=2686976 req_one=1
nbdkit: file.5: debug: file: pread count=458752 offset=2686976
nbdkit: file.5: debug: file: extents count=983040 offset=3211264 req_one=1
nbdkit: file.6: debug: file: pread count=983040 offset=3211264
nbdkit: file.7: debug: file: extents count=2031616 offset=4259840 req_one=1
nbdkit: file.8: debug: file: pread count=2031616 offset=4259840
nbdkit: file.9: debug: file: extents count=11599872 offset=6356992 req_one=1
nbdkit: file.10: debug: file: pread count=2097152 offset=6356992
nbdkit: file.11: debug: file: pread count=2097152 offset=8454144
...
nbdkit: file.14: debug: file: extents count=393216 offset=5591269376 req_one=1
nbdkit: file.6: debug: file: pread count=393216 offset=5591269376
nbdkit: file.6: debug: file: extents count=313851904 offset=5591728128 req_one=1
nbdkit: file.4: debug: file: pread count=1048576 offset=5591728128
nbdkit: file.4: debug: file: extents count=312803328 offset=5592776704 req_one=1
nbdkit: file.8: debug: file: extents count=535232512 offset=5905580032 req_one=1
nbdkit: file.3: debug: file: extents count=524288 offset=6441402368 req_one=1
nbdkit: file.13: debug: client sent NBD_CMD_DISC, closing connection

Total 960 calls per one import.

A tiny delay in getting extents can slow down the entire import.

Nir