[Libguestfs] [libnbd] Simultaneous read and write

[Richard W.M. Jones]

I think it's possible to do this without making any large changes to
the implementation of libnbd.

The idea is to have a new struct socket subtype:

  https://github.com/libguestfs/libnbd/blob/e63a11736930c381a79a8cc2d03844cfff5db3ef/lib/internal.h#L190

which will work like the current raw struct socket for send():

  https://github.com/libguestfs/libnbd/blob/e63a11736930c381a79a8cc2d03844cfff5db3ef/lib/socket.c#L32

but for recv the data will come from a buffer populated by the caller
using their own thread.  The caller will be required to set up a
second thread which does basically:

  pthread_create (start_second_thread);

  start_second_thread ()
  {
    int fd = nbd_aio_get_fd (nbd);
    char buf[BUFSIZ];
    for (;;) {
      poll (fd);
      len = recv (fd, buf, sizeof buf, 0);
      nbd_aio_threaded_read (fd, buf, len);
    }
  }

Note this doesn't require any changes to the state machine, and
probably not any changes to the existing main loop.  The state machine
still runs through the same states as before, it's just that when it's
reading from the socket, instead of those reads coming from calls to
raw recv(), instead they will be satisfied immediately from the data
supplied by the second thread and buffered up in the handle.  It
should also be compatible with TLS, which is currently implemented by
adding a second struct socket layer over the raw struct socket, but in
this case would work by adding the same layer over the threaded read
struct socket.

The actual mechanics of this are a bit more complex than described:

 - We will need the caller to tell libnbd that we want to use the
   handle in this way, requiring a function such as
   nbd_set_threaded_read() to be called before connection.  The
   purpose of this is to ensure our connect calls set up a threaded
   read struct socket instead of the normal raw struct socket.

 - We may still need a pipe-to-self.  I'm not sure if this is really
   necessary, but if so nbd_set_threaded_read can return a pipe fd
   which the caller will have to poll for POLLIN in the main loop.

 - I'm not sure what the end condition is for the loop in
   start_second_thread.  Probably nbd_aio_threaded_read returns an
   indication to exit the thread.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW