[Libguestfs] [nbdkit PATCH] crypto: Tweak handling of SEND_MORE

[Eric Blake]

On 6/10/19 5:11 PM, Eric Blake wrote:
> In the recent commit 3842a080 to add SEND_MORE support, I blindly
> implemented the tls code as:
>  if (SEND_MORE) {
>    cork
>    send
>  } else {
>    send
>    uncork
>  }
> because it showed improvements for my test case of aio-parallel-load
> from libnbd. But that test sticks to 64k I/O requests.
> 
...
> (which may or may not happen, based on how much overhead TLS adds,
> given that 64k is the maximum TCP segment).  It may turn out that we
> can tune the limit for slightly better performance (maybe 32k is
> smarter than 64k), but since the previous commit saw improvement with
> the benchmark using 64k packets, that's the value picked for now.

After more measurements, and a tweak to libnbd aio-parallel-load to make
it easier to test various command sizes, I came up with this rough table:

      packet size:  512             32k             64k             2m
    No cork [1]:    325,209.3       187,598.9       116,304.2       4,856.7
    full cork [2]:  461,083.8       198,009.3       118,072.2       4,595.1
    32k cork:       456,365.5       196,586.5       117,772.4       4,779.1
    64k cork [3]:   450,443.6       194,505.9       117,226.6       4,822.2
    128k cork:      455,049.4       197,991.4       117,157.9       4,953.7

    [1] Using the code prior to 3842a080
    [2] Using the code from 3842a080
    [3] The value chosen for this patch

Some of those numbers are in the noise, but I'm feeling more confident
about my choice, and the fact that uncorking early DOES benefit large
payload sizes without too much penalty to small payload sizes.  So I'm
going ahead and pushing this patch with one additional tweak:


> @@ -357,7 +363,11 @@ crypto_send (struct connection *conn, const void *vbuf, size_t len, int flags)
> 
>    assert (session != NULL);
> 
> -  if (flags & SEND_MORE)
> +  if (len >= MAX_SEND_MORE_LEN) {

Here, I'm now using

 if (len + gnutls_record_check_corked (session) > MAX_SEND_MORE_LEN) {

after verifying in the gnutls source code that blindly checking the
corked length is not a significant burden (no syscalls).

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/libguestfs/attachments/20190612/76f73668/attachment.sig>