[dm-devel] Thoughts about multipathd's log thread

[Benjamin Marzinski]

On Mon, Nov 02, 2020 at 01:17:28PM +0100, Martin Wilck wrote:
> (sending again, with dm-devel on cc. Sorry!)
> 
> Hi Ben, hi Christophe,
> 
> AFAIU, the purpose of the log thread is to avoid blocking while writing
> log messages to the syslog socket. The thread has been in place for a
> long time. However, the large majority of multipath installations today
> runs on systemd-enabled systems, where we don't use the log thread.
> Rather, we write to stderr (which is actually a journald socket)
> directly.
> 
> That makes me wonder: if the problem of blocking while writing to the
> log socket is real, it should happen with journald as well (possibly
> more so than with syslogd; journald doesn't exactly have the reputation
> of being highly efficient). Thus, we should be using the log thread
> also for the systemd case

It does seem sort of wasteful to me to have a thread buffering writes to
a daemon that itself buffers writes. If journald is blocking logging
processes from continuing, that's a big problem, and not just for
multipath. Obviously multipathd, or at the very least, path checking, is
supposed to continue working when there is no access to the root
filesystem, which is a stricter requirement than other programs have.
But IO delays happen, and journald better be decoupling them from the
logging process. I have seen journald drop log messages, presumably
because it isn't just blocking everything that wants to log to it.

>  OTOH, I'm not aware of any bug reports
> saying that multipathd wasn't able to handle events because of blocking
> log calls. So perhaps the issue is rather theoretical? In that case, we
> could do away with all the complexity the log thread involves.
> 
> What do you think?

I do believe that syslog is allowed to block the caller, but I agree
that we've mostly moved on to a systemd world where multipathd is
writing to stderr. Removing this will make multipathd run a real risk of
hanging on logging when not run through systemd. I just don't know how
likely that scenario is anymore.

-Ben

> Martin
> 
>