[PATCH] rpc: Temporarily stop accept()-ing new clients on EMFILE

[Daniel P. Berrangé]

On Tue, Oct 12, 2021 at 02:41:39PM +0200, Michal Privoznik wrote:
> This commit is related to 5de203f879 which I pushed a few days
> ago. While that commit prioritized closing clients socket over
> the rest of I/O process, this one goes one step further and
> temporarily suspends processing new connection requests.
> 
> A brief recapitulation of the problem:
> 
> 1) assume that libvirt is at the top of RLIMIT_NOFILE (that is no
>    new FDs can be opened).
> 
> 2) we have a client trying to connect to a UNIX/TCP socket
> 
> Because of 2) our event loop sees POLLIN on the socket and thus
> calls virNetServerServiceAccept(). But since no new FDs can be
> opened (because of 1)) the request is not handled and we will get
> the same event on next iteration. The poll() will exit
> immediately because there is an event on the socket.  Thus we end
> up in an endless loop.
> 
> To break the loop and stop burning CPU cycles we can stop
> listening for events on the socket and set up a timer tho enable
> listening again after some time (I chose 5 seconds because of no
> obvious reason).
> 
> There's another area where we play with temporarily suspending
> accept() of new clients - when a client disconnects and we check
> max_clients against number of current clients. Problem here is
> that max_clients can be orders of magnitude larger than
> RLIMIT_NOFILE but more importantly, what this code considers
> client disconnect is not equal to closing client's FD.
> A client disconnecting means that the corresponding client
> structure is removed from the internal list of clients. Closing
> of the client's FD is done from event loop - asynchronously.
> 
> To avoid this part stepping on the toes of my fix, let's make the
> code NOP if socket timer (as described above) is active.
> 
> Signed-off-by: Michal Privoznik <mprivozn at redhat.com>
> ---
>  src/libvirt_remote.syms       |  1 +
>  src/rpc/virnetserver.c        |  9 ++++++
>  src/rpc/virnetserverservice.c | 53 ++++++++++++++++++++++++++++++++++-
>  src/rpc/virnetserverservice.h |  2 ++
>  src/rpc/virnetsocket.c        | 15 ++++++++++
>  5 files changed, 79 insertions(+), 1 deletion(-)

Reviewed-by: Daniel P. Berrangé <berrange at redhat.com>


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|