[Libguestfs] Two multi-conn questions

Richard W.M. Jones rjones at redhat.com
Wed Feb 1 12:40:45 UTC 2023


Hi Eric,

A couple of semi-related multi-conn questions.

(1) nbdkit-curl-plugin does not currently advertise multi-conn.  However
forcing multi-conn makes it faster, eg:

$ time nbdkit -r curl https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img timeout=2000 --run ' nbdcopy --no-extents -p $uri jammy-server-cloudimg-amd64.img '
█ 100% [****************************************]

real   1m15.438s
user   0m2.513s
sys    0m4.063s

$ time nbdkit -r curl https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img --filter=multi-conn multi-conn-mode=unsafe timeout=2000 --run ' nbdcopy --no-extents -p $uri jammy-server-cloudimg-amd64.img '
█ 100% [****************************************]

real   0m59.635s
user   0m3.010s
sys    0m4.538s

(Note that this is line speed - wget also takes 59 seconds)

I believe that multi-conn should be safe to enable in the curl plugin,
certainly if we're in read-only mode (which is the most common case
for this plugin).  It's probably not safe when operating in read-write
mode, because HTTP doesn't have any concept of flushing, but almost no
web server supports writing.

(2) qemu's NBD client still doesn't seem to support multi-conn.  This
makes qemu-img convert very slow when reading from nbdkit, as in the
example below.  (Note the '.img' file on that website is actually a
qcow2 file.)

$ time nbdkit -r curl https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img --filter=multi-conn multi-conn-mode=unsafe timeout=2000 --run ' qemu-img convert -W -m 16 -p -f qcow2 $uri -O raw jammy-server-cloudimg-amd64.img.raw '
    (100.00/100%)

real	3m29.412s
user	0m17.856s
sys	0m8.218s

Since we know from (1) above that copying could go at line speed when
multi-conn is enabled, and also that qemu-img convert from the same
local file takes only about 2 seconds, it seems like qemu-img convert
above could go a lot faster if we allowed multi-conn.

Is there any work on adding multi-conn support to qemu's NBD client?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top


More information about the Libguestfs mailing list