[Libguestfs] More parallelism in VDDK driver (was: Re: CFME-5.11.7.3 Perf. Tests)

[Nir Soffer]

On Wed, Aug 5, 2020 at 3:28 PM Richard W.M. Jones <rjones at redhat.com> wrote:
>
>
> Nir, BTW what are you using for performance testing?

virt-v2v with local image, or imageio client with local image.

> As far as I can tell it's not possible to make qemu-img convert use
> multi-conn when connecting to the source (which is going to be a
> problem if we want to use this stuff in virt-v2v).

But do we need multiple connections? qemu can send multiple requests
on one connection.

Did you try to copy an image from nbdkit file plugin to another nbdkit
file plugin using qemu-img convert?

    nbdkit file pluing -> qemu-img convert -W nbd:///?socket=src.sock
nbd:///?socket=dst.sock-> nbdkit file plugin

I did not try it but I will be surprised if we don't get all 8 threads
busy in both sides

The reason we use multiple connection in imageio is that we don't
support async I/O in
http client, http server, and nbd client, and it is much easier to
open new connection with
the entire stuck compared to rewriting the http server and nbd client.

It also much hard to provide easy to use interface for users
supporting async I/O.

> Instead I've hacked up a copy of this program from libnbd:
>
> https://github.com/libguestfs/libnbd/blob/master/examples/threaded-reads-and-writes.c
>
> so that it only does reads and aligns requests to 512 bytes.
>
> At least this is testing multi-conn, but there should be an easier way ...
>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> virt-top is 'top' for virtual machines.  Tiny program with many
> powerful monitoring features, net stats, disk stats, logging, etc.
> http://people.redhat.com/~rjones/virt-top
>