[Libguestfs] nbdcpy: from scratch nbdcopy using io_uring

[Richard W.M. Jones]

On Sun, Jun 27, 2021 at 05:00:00PM +0530, Abhay Raj Singh wrote:
> I ran into a problem while working on receiving data from nbd source
> (Reply to NBD_CMD_READ)
> 
> As you know we need to parse the error in Reply Header before we can
> proceed reading the data.
> Let's say an error occurred so instead of
> HEADER1_DATA1..._HEADER2_DATA2... we will
> get HEADER1_HEADER2_ DATA2... (as far as I know) so submitting a recv
> request to io_uring
> with length = sizeof(HEADER1+DATA1) would cause problem as it won't
> detect NBD packet boundaries
> and will give us as many bytes we ask it (I may be wrong here that's
> what I read till now).
>
> A remedy to this would be just submit 'header reads' to io_uring when
> we get a read and if header says there were no errors
> we can be sure there is length bytes ready to be read in the
> buffer(rest of the NBD packet) and read won't block.
> Now, as far as I can tell this would work as I expect but our main
> concern is avoiding copy_user_enhanced_fast_string
> so this won't be nice.

IIUC there are two separate concerns here:

(1) How do we process errors sent by the NBD source side.

(2) How do we copy data from the source side to the target side
without copying it through userspace.

Note that NBD headers are very small, just a few bytes.  We have to
copy those to userspace in order to examine them (eg. to check for
errors, but also to check for the command sequence number).  But
they're small so copying them to userspace is probably no problem.

We should still be able to avoid copying all the data through
userspace though.  Data is much bigger than headers.

> Also attaching metadata (Operation) to read SQE doesn't make sense
> because As far as I know io_uring won't be able to tell
> the difference the read is for which io_uring request, Reply Header's
> handle will tell us which operation in operations vector
> does this NBD packet belong to.
> 
> Another solution would be opening multiple sockets one for each slot
> in operations vector, only one NBD operation runs on a socket
> i.e. only one inflight request per socket, that too sounds like a bad idea.

Yes that doesn't sound like it can be right.

There must be other programs that we can look at which use io_uring to
parse network traffic.  Is there (eg) a web client that uses io_uring?
Web servers also send a sequence of header, data, header, ... so we
might be able to learn lessons there.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW