[dm-devel] [PATCH 33/35] multipathd: uxlsnr: use poll loop for sending, too

[Benjamin Marzinski]

On Thu, Sep 16, 2021 at 11:33:29AM +0200, Martin Wilck wrote:
> On Wed, 2021-09-15 at 23:22 -0500, Benjamin Marzinski wrote:
> > On Fri, Sep 10, 2021 at 01:41:18PM +0200, mwilck at suse.com wrote:
> > > From: Martin Wilck <mwilck at suse.com>
> > > 
> > > send_packet() may busy-loop. By polling for POLLOUT, we can
> > > avoid that, even if it's very unlikely in practice.
> > > 
> > > Signed-off-by: Martin Wilck <mwilck at suse.com>
> > > ---
> > >  multipathd/uxlsnr.c | 39 ++++++++++++++++++++++++++++++++-------
> > >  1 file changed, 32 insertions(+), 7 deletions(-)
> > > 
> > > diff --git a/multipathd/uxlsnr.c b/multipathd/uxlsnr.c
> > > index 1bf4126..c18b2c4 100644
> > > --- a/multipathd/uxlsnr.c
> > > +++ b/multipathd/uxlsnr.c
> > > @@ -588,15 +588,37 @@ static void handle_client(struct client *c,
> > > struct vectors *vecs, short revents)
> > >                 if (get_strbuf_len(&c->reply) == 0)
> > >                         default_reply(c, c->error);
> > >  
> > > -               const char *buf = get_strbuf_str(&c->reply);
> > > +               if (c->cmd_len == 0) {
> > > +                       size_t len = get_strbuf_len(&c->reply) + 1;
> > >  
> > > -               if (send_packet(c->fd, buf) != 0)
> > > -                       dead_client(c);
> > > -               else
> > > -                       condlog(4, "cli[%d]: Reply [%zu bytes]", c-
> > > >fd,
> > > -                               get_strbuf_len(&c->reply) + 1);
> > > -               reset_strbuf(&c->reply);
> > > +                       if (send(c->fd, &len, sizeof(len),
> > > MSG_NOSIGNAL)
> > > +                           != sizeof(len)) {
> > 
> > This assumes that failing to send the size is always an error. What
> > about if we get EINTR/EAGAIN? Also, it seems pretty likely that we will
> > either send all of the size or none of it, but I'm not sure we can
> > guarantee that.  send_packet() handled partitial writes of the length.
> > Actually, mpath_recv_reply_len() which is still used by CLT_RECV still
> > uses read_all(), instead of just polling again on partial reads.
> 
> That was intentional. I couldn't imagine that reading or writing 8
> bytes would block (after all, we received POLLIN / POLLOUT for the
> socket). Note the minimum socket buffer size that the kernel uses is
> larger than 2048 bytes.
> 
> https://elixir.bootlin.com/linux/latest/source/include/net/sock.h#L2339
> 
> When we send the size value, the send buffer is empty by definition
> (even for interactive sessions, if the client uses libmpathcmd, it will
> have fetched the response before sending new requests). Likewise for
> receiving, the size will be received in one piece if the client uses
> libmpathcmd.
> 
> I take it that some malicious or badly designed client could access the
> socket bypassing libmpathcmd, sending or receiving the size byte-by-
> byte, or in other strange patterns. But I think we are within our
> rights to error out in such cases; it may actually be the right thing
> to do that, as we don't really want to service clients doing tricks
> like that. I will do some tests.

In this case, calling mpath_recv_reply_len() is probably overkill.
 
> > Also, the fd is not set to be non-blocking. and if we fall through to
> > CLT_SEND, we haven't checked for a POLLOUT revent, so technically, I
> > believe the write could block here.
> 
> Right, we have to do that. Thanks for pointing it out. This also means
> that polling in read_all() and send_packet() was pointless in the first
> place ...
> 
> Thanks!
> Martin
> 
> PS: I was wondering why we haven't been using SOCK_DGRAM for the
> multipath socket in the first place. It would have avoided all this
> hassle...

That design decision predates me working on the code.

-Ben

>