[Libguestfs] [PATCH nbdkit] Add new filter for rate-limiting connections.

[Richard W.M. Jones]

On Fri, Mar 08, 2019 at 06:38:10AM -0600, Eric Blake wrote:
> On 3/5/19 4:38 AM, Richard W.M. Jones wrote:
> > ---
> >  filters/delay/nbdkit-delay-filter.pod |   4 +-
> >  filters/rate/nbdkit-rate-filter.pod   |  84 +++++++++
> >  configure.ac                          |   2 +
> >  filters/rate/bucket.h                 |  62 +++++++
> >  filters/rate/bucket.c                 | 173 +++++++++++++++++++
> >  filters/rate/rate.c                   | 235 ++++++++++++++++++++++++++
> >  TODO                                  |   9 +
> >  filters/rate/Makefile.am              |  64 +++++++
> >  tests/Makefile.am                     |   6 +-
> >  tests/test-rate.sh                    |  60 +++++++
> >  10 files changed, 697 insertions(+), 2 deletions(-)
> 
> I see I was too slow in reviewing this before you pushed (that's okay,
> overall it looks reasonable).
> 
> 
> > +
> > +=item nbdkit --filter=rate memory size=64M connection-rate=50K rate=1M
> > +
> > +Limit each connection to S<50 Kbps>.  Additionally the total bandwidth
> > +across all connections to the server is limited to S<1 Mbps>.
> 
> If there are more than 40 clients, does this setup fairly service all of
> them at a lower per-client rate?

Probably it doesn't.  It just does the simple/stupid thing of checking
the global bucket first and the per-connection bucket second.
Presumably in the overloaded case the global bucket would be heavily
contended, meaning that it would depend on how threads are scheduled,
but I would expect to see some threads starving.

> > +=head1 NOTES
> > +
> > +The rate filter approximates the bandwidth used by the NBD protocol on
> > +the wire.  Some operations such as zeroing and trimming are
> > +effectively free (because only a tiny NBD message is sent over the
> > +network) and so do not count against the bandwidth limit.  NBD and TCP
> > +protocol overhead is not included, so you may find that other tools
> > +such as L<tc(8)> and L<iptables(8)> give more accurate results.
> > +
> > +There are separate bandwidth limits for read and write (ie. upload and
> > +download to the server).
> > +
> 
> Is it worth mentioning that the blocksize filter can be used to ensure
> smoother spreading of the bandwidth? (If a large client request is
> broken into 128k max packets, that hits the network in a smoother
> pattern than a single request).

Yes, I've added a commit to mention this:

https://github.com/libguestfs/nbdkit/commit/b165ff3ef6c20f62108bafa2834cd849fba2a695

> > +++ b/TODO
> > @@ -129,6 +129,15 @@ Suggestions for filters
> >  * nbdkit-cache-filter should handle ENOSPC errors automatically by
> >    reclaiming blocks from the cache
> >  
> > +nbdkit-rate-filter:
> > +
> > +* allow other kinds of traffic shaping such as VBR
> > +
> > +* limit traffic per client (ie. per IP address)
> > +
> > +* split large requests to avoid long, lumpy sleeps when request size
> > +  is much larger than rate limit
> 
> Can't you pair the blocksize filter in front of this one to accomplish
> that already?

Yes, but that involves the user taking deliberate action.  It should
also be possible to split up requests transparently in the rate
filter.  Note that the filter has unique information here because it
knows how long it will sleep for given the request block size and
rate, which for the user of nbdkit involves a complex calculation or
tuning.

However by mentioning the blocksize filter in the man page (previous
point) that does give the user the option to do this.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/