[dm-devel] [PATCH 33/35] multipathd: uxlsnr: use poll loop for sending, too

[Benjamin Marzinski]

On Fri, Sep 10, 2021 at 01:41:18PM +0200, mwilck at suse.com wrote:
> From: Martin Wilck <mwilck at suse.com>
> 
> send_packet() may busy-loop. By polling for POLLOUT, we can
> avoid that, even if it's very unlikely in practice.
> 
> Signed-off-by: Martin Wilck <mwilck at suse.com>
> ---
>  multipathd/uxlsnr.c | 39 ++++++++++++++++++++++++++++++++-------
>  1 file changed, 32 insertions(+), 7 deletions(-)
> 
> diff --git a/multipathd/uxlsnr.c b/multipathd/uxlsnr.c
> index 1bf4126..c18b2c4 100644
> --- a/multipathd/uxlsnr.c
> +++ b/multipathd/uxlsnr.c
> @@ -588,15 +588,37 @@ static void handle_client(struct client *c, struct vectors *vecs, short revents)
>  		if (get_strbuf_len(&c->reply) == 0)
>  			default_reply(c, c->error);
>  
> -		const char *buf = get_strbuf_str(&c->reply);
> +		if (c->cmd_len == 0) {
> +			size_t len = get_strbuf_len(&c->reply) + 1;
>  
> -		if (send_packet(c->fd, buf) != 0)
> -			dead_client(c);
> -		else
> -			condlog(4, "cli[%d]: Reply [%zu bytes]", c->fd,
> -				get_strbuf_len(&c->reply) + 1);
> -		reset_strbuf(&c->reply);
> +			if (send(c->fd, &len, sizeof(len), MSG_NOSIGNAL)
> +			    != sizeof(len)) {

This assumes that failing to send the size is always an error. What
about if we get EINTR/EAGAIN? Also, it seems pretty likely that we will
either send all of the size or none of it, but I'm not sure we can
guarantee that.  send_packet() handled partitial writes of the length.
Actually, mpath_recv_reply_len() which is still used by CLT_RECV still
uses read_all(), instead of just polling again on partial reads.

Also, the fd is not set to be non-blocking. and if we fall through to
CLT_SEND, we haven't checked for a POLLOUT revent, so technically, I
believe the write could block here.

-Ben

> +				c->error = -ECONNRESET;
> +				return;
> +			}
> +			c->cmd_len = len;
> +			return;
> +		}
>  
> +		if (c->len < c->cmd_len) {
> +			const char *buf = get_strbuf_str(&c->reply);
> +
> +			n = send(c->fd, buf + c->len, c->cmd_len, MSG_NOSIGNAL);
> +			if (n == -1) {
> +				if (errno == EAGAIN || errno == EINTR)
> +					return;
> +				else {
> +					c->error = -ECONNRESET;
> +					return;
> +				}
> +			}
> +			c->len += n;
> +		}
> +		if (c->len < c->cmd_len)
> +			/* continue polling */
> +			return;
> +
> +		condlog(4, "cli[%d]: Reply [%zu bytes]", c->fd, c->cmd_len);
>  		set_client_state(c, CLT_RECV);
>  		break;
>  
> @@ -701,6 +723,9 @@ void *uxsock_listen(long ux_sock, void *trigger_data)
>                          case CLT_RECV:
>                                  polls[i].events = POLLIN;
>                                  break;
> +			case CLT_SEND:
> +				polls[i].events = POLLOUT;
> +				break;
>                          default:
>  				/* don't poll for this client */
>                                  continue;
> -- 
> 2.33.0