[dm-devel] Thoughts about multipathd's log thread

[Benjamin Marzinski]

On Wed, Nov 04, 2020 at 03:11:04PM +0100, Hannes Reinecke wrote:
> On 11/2/20 11:40 PM, Benjamin Marzinski wrote:
> >On Mon, Nov 02, 2020 at 01:17:28PM +0100, Martin Wilck wrote:
> >>(sending again, with dm-devel on cc. Sorry!)
> >>
> >>Hi Ben, hi Christophe,
> >>
> >>AFAIU, the purpose of the log thread is to avoid blocking while writing
> >>log messages to the syslog socket. The thread has been in place for a
> >>long time. However, the large majority of multipath installations today
> >>runs on systemd-enabled systems, where we don't use the log thread.
> >>Rather, we write to stderr (which is actually a journald socket)
> >>directly.
> >>
> >>That makes me wonder: if the problem of blocking while writing to the
> >>log socket is real, it should happen with journald as well (possibly
> >>more so than with syslogd; journald doesn't exactly have the reputation
> >>of being highly efficient). Thus, we should be using the log thread
> >>also for the systemd case
> >
> >It does seem sort of wasteful to me to have a thread buffering writes to
> >a daemon that itself buffers writes. If journald is blocking logging
> >processes from continuing, that's a big problem, and not just for
> >multipath. Obviously multipathd, or at the very least, path checking, is
> >supposed to continue working when there is no access to the root
> >filesystem, which is a stricter requirement than other programs have.
> >But IO delays happen, and journald better be decoupling them from the
> >logging process. I have seen journald drop log messages, presumably
> >because it isn't just blocking everything that wants to log to it.
> >
> >>  OTOH, I'm not aware of any bug reports
> >>saying that multipathd wasn't able to handle events because of blocking
> >>log calls. So perhaps the issue is rather theoretical? In that case, we
> >>could do away with all the complexity the log thread involves.
> >>
> >>What do you think?
> >
> >I do believe that syslog is allowed to block the caller, but I agree
> >that we've mostly moved on to a systemd world where multipathd is
> >writing to stderr. Removing this will make multipathd run a real risk of
> >hanging on logging when not run through systemd. I just don't know how
> >likely that scenario is anymore.
> >
> Well ... isn't that what the option '-d' is for?
> Namely _not_ starting the log thread when running under systemd?

Martin is arguing that syslogd is at least as likely not to block as
journald, so if we don't need the log thread when writing to journald
(though stderr), we also don't need the log thread when writing to
syslogd. Correct me, if I'm wrong Martin.

-Ben

> 
> Cheers,
> 
> Hannes
> -- 
> Dr. Hannes Reinecke                Kernel Storage Architect
> hare at suse.de                              +49 911 74053 688
> SUSE Software Solutions GmbH, Maxfeldstr. 5, 90409 Nürnberg
> HRB 36809 (AG Nürnberg), Geschäftsführer: Felix Imendörffer