[Libguestfs] [PATCH libnbd] copy: Always allow multiple connections

[Richard W.M. Jones]

On Thu, May 27, 2021 at 02:13:24PM +0300, Nir Soffer wrote:
> On Thu, May 27, 2021 at 1:49 PM Richard W.M. Jones <rjones at redhat.com> wrote:
> >
> >
> > Actually no, I had to revert this.
> >
> > https://gitlab.com/nbdkit/libnbd/-/commit/fb42056929e9177e0de2603e2f430a8111ae7a5f
> >
> > I added a test:
> >
> > https://gitlab.com/nbdkit/libnbd/-/commit/23b41ad0a2a6d6582a8e643e42960d2e3c72e53e
> >
> > However the test deadlocks when we try to open multiple connections to
> > qemu-nbd (with your patch).  As best I can tell this is because of
> > qemu's file locking.  qemu-nbd doesn't seem to support the -U option
> > found in qemu-img, but even if it did I don't think it would be a good
> > idea to tell users that they have to use it.
> 
> When using qemu-nbd you need to specify --shared=4 on both source and
> destination.

Ah, I didn't realise that.  So my explanation for why it failed was
wrong.

It has to be said qemu-nbd's defaults are a pain, especially having to
always specify the -t option.

> Current nbdcopy depends on the server supporting multiple
> connections, so we try to open the additional connections in the
> main thread. If the server supports only one connection (--shared=4
> not specified), this will deadlock.
>
> One way to fix this is to open the extra connection in the worker
> threads. If the server does not support more then one connection,
> only one worker can make progress, and the other will just block on
> nbd_connect(). When the first worker finishes all the work, the rest
> of the workers will connect and find that they have no work to do
> and exit.

Agreed.

> > I think this needs to be fixed in qemu-nbd itself.
> 
> I don't think we can change qemu-nbd behavior at this point.

IIUC the reason why qemu-nbd doesn't support multi-conn is because of
the generality of the qemu block layer.  If qemu-nbd is backed by --
for example -- a Ceph device then we'd need to check if Ceph was safe
for multi-conn.

However I wonder if it's possible to advertise multi-conn in some
limited scenarios.  Could qemu-nbd check if the backing block device
is a plain raw file (or other limited cases) and offer multi-conn
then?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org