[Libguestfs] [PATCH nbdkit] Add new filter for rate-limiting connections.

[Eric Blake]

On 3/5/19 4:38 AM, Richard W.M. Jones wrote:
> ---
>  filters/delay/nbdkit-delay-filter.pod |   4 +-
>  filters/rate/nbdkit-rate-filter.pod   |  84 +++++++++
>  configure.ac                          |   2 +
>  filters/rate/bucket.h                 |  62 +++++++
>  filters/rate/bucket.c                 | 173 +++++++++++++++++++
>  filters/rate/rate.c                   | 235 ++++++++++++++++++++++++++
>  TODO                                  |   9 +
>  filters/rate/Makefile.am              |  64 +++++++
>  tests/Makefile.am                     |   6 +-
>  tests/test-rate.sh                    |  60 +++++++
>  10 files changed, 697 insertions(+), 2 deletions(-)

I see I was too slow in reviewing this before you pushed (that's okay,
overall it looks reasonable).


> +
> +=item nbdkit --filter=rate memory size=64M connection-rate=50K rate=1M
> +
> +Limit each connection to S<50 Kbps>.  Additionally the total bandwidth
> +across all connections to the server is limited to S<1 Mbps>.

If there are more than 40 clients, does this setup fairly service all of
them at a lower per-client rate?


> +=head1 NOTES
> +
> +The rate filter approximates the bandwidth used by the NBD protocol on
> +the wire.  Some operations such as zeroing and trimming are
> +effectively free (because only a tiny NBD message is sent over the
> +network) and so do not count against the bandwidth limit.  NBD and TCP
> +protocol overhead is not included, so you may find that other tools
> +such as L<tc(8)> and L<iptables(8)> give more accurate results.
> +
> +There are separate bandwidth limits for read and write (ie. upload and
> +download to the server).
> +

Is it worth mentioning that the blocksize filter can be used to ensure
smoother spreading of the bandwidth? (If a large client request is
broken into 128k max packets, that hits the network in a smoother
pattern than a single request).


> +++ b/TODO
> @@ -129,6 +129,15 @@ Suggestions for filters
>  * nbdkit-cache-filter should handle ENOSPC errors automatically by
>    reclaiming blocks from the cache
>  
> +nbdkit-rate-filter:
> +
> +* allow other kinds of traffic shaping such as VBR
> +
> +* limit traffic per client (ie. per IP address)
> +
> +* split large requests to avoid long, lumpy sleeps when request size
> +  is much larger than rate limit

Can't you pair the blocksize filter in front of this one to accomplish
that already?

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/libguestfs/attachments/20190308/256a75e9/attachment.sig>