[Libguestfs] [libnbd] GSoC21: IO_uring optimization questions

[Richard W.M. Jones]

[Adding Stefan: Have we retro-fitted io_uring into any other libraries?]

Welcome Abhay,

On Tue, Mar 30, 2021 at 09:33:17PM +0530, Abhay Raj Singh wrote:
> Greetings everyone! I am Abhay a 3rd-year student at the
> National Institute of Technology, Hamirpur, Bharat(India).
> 
> I am interested in utilizing the recently introduced feature
> io_uring of the Linux kernel to speed up libnbd async I/O.
> 
> I have been in contact with Richard regarding this and
> thanks to his guidance I have some ideas and approaches.
> 
> The rough approaches I can think of are:
> 1. Replacing the poll mechanism(nbd_poll) with the io_uring one.

This would be nice and self-contained.

> 2. A socket wrapper using io_uring
> 
> The latter requires less work as far as I can see and is easily
> reusable. Though I have only recently read about the socket
> wrapper method so can't tell for sure. I will try making a
> wrapper to check feasibility.

Does "socket wrapper" mean adding another struct socket_ops defined in
lib/internal.h and implemented so far in lib/socket.c (for plain
sockets) and lib/crypto.c (for TLS sockets)?

As an aside, libnbd only makes socket calls in a few places:

$ git grep -- '->recv'
generator/states-reply.c:  r = h->sock->ops->recv (h, h->sock, h->rbuf, h->rlen);
generator/states-reply.c:    /* sock->ops->recv called set_error already. */
generator/states.c:  r = h->sock->ops->recv (h, h->sock, rbuf, rlen);
generator/states.c:    /* sock->ops->recv called set_error already. */

$ git grep -- '->send'
generator/states.c:  r = h->sock->ops->send (h, h->sock, h->wbuf, h->wlen, h->wflags);
generator/states.c:    /* sock->ops->send called set_error already. */

(Also you'll want to look for '->pending', '->get_fd',
'->shut_writes', '->close').

> A general suggestion for io_uring is to not use it, directly:)
> But use a convenient wrapper liburing instead.
> 
> The doubts I have regarding this are:
> 1.
> Should I create a new API or replace the old aio API.
> On one end I see no problem in doing so but I am
> not familiar with uses

If there's a new API then that's going to require every existing user
to opt in, which isn't nice for those users.  They would need to
choose io_uring (and risk not working on many platforms) or try to
support both APIs.  So the bar to doing this way must be very high -
only if it's not possible any other way.

> 2.
> For testing wouldn't libnbd limited by nbdkit if try
> to test on my pc and would be only marginally better
> unless nbdkit is ported too.

You could also test against qemu-nbd, although it is also not using
io_uring AFAIK.

Also you could test against a simple hand-written NBD server
implemented using io_uring.

Or maybe there will be enough of a performance difference against
unmodified nbdkit.

> As the task is not too difficult once I get my head around
> the code base, I might be able to port nbdkit too(incase of socket wrapper)
> and write some tests in the 180-200hr period of GSoC.
> 
> Thanks!
> Humble Regards,
> Abhay
> (rathod-sahaab)

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org