[dm-devel] [PATCH 17/33] multipathd: imply -n if find_multipaths is set

[Martin Wilck]

On Wed, 2017-04-05 at 18:03 -0500, Benjamin Marzinski wrote:
> On Tue, Feb 28, 2017 at 05:23:13PM +0100, Martin Wilck wrote:
> > Automatic detection of new devices with find_multipaths
> > doesn't work correctly currently. Therefore, for now,
> > imply ignore_new_devs if find_multipaths is seen.
> 
> I would rather not do this (at least outside of the initramfs), since
> it
> keeps multipathd from automatically creating multipath devices as
> expected when you enable find_multipaths. I admit that these path
> devices won't be correctly claimed in udev when they appear for the
> first time, but it's hard to believe that they are critical the very
> first time they appear on the system. I have a patch that I can send
> upstream that triggers a change uevent on path devices when they are
> added to the wwids file.  This means that these devices will be
> correctly claimed by multipath as soon as it gets set up on top of
> them.
> 
> Martin, what do you thing about reverting this change and triggering
> a
> uevent instead?

I haven't been able to obtain stable behavior in my experiments, that's
why I made this patch. The problem is the correspondence between the
multipath invocation in 56-multipath.rules and multipathd. I explained
the problems I had in more detail in the commit message of patch 16 of
the series. 

[Note: SUSE uses "-i" in multipath.rules, whereas Fedora/RH does not.
However with patch 16/33 I changed the code such that "-i" is ignored
when find_multipaths is used, so that the multipath call in 56-
multipath.rules actually does the same in both distros in the
find_multipaths case (only considers paths from the WWIDs file).]

For a path that's not in the WWIDs file yet, the udev rule will always
return FALSE (better than with -i, where the result would be
essentially random). That means that the path will be further processed
by udev and systemd, and file systems will be mounted, LVM PVs scanned,
etc.

When multipathd tries to grab this path later, it will most likely face
an EBUSY error. Triggering another uevent won't change much if the
devices are already in use. In the worst case, the multipath map will
be non-functional but the paths will get SYSTEMD_READY=0 set in the new
uevent, so that the device might become non-accessible by the system
either through dm (map not set up successfully) or directly
(SYSTEMD_READY=0).

Doing this right is a hard problem IMO. Hannes and I thought about some
new means of communication between multipath and multipathd that would
guarantee that udev rules and multipathd were in agreement about every
path, something fast and (almost) lock-less like shared memory. We have
no code yet, though. (Note: implementing this would require reverting
our patch "multipathd: start daemon after udev trigger").

One other idea I had was to have the udev rule treat every path as a
multipath device path in the first place, then wait for a certain
amount of time whether additional paths are detected. If yes, maps will
be set up by multipathd and all is good. If no, we'd need to re-trigger 
uevents for all "single-path" devices after the timeout, and now we'd
distinguish between multipath and non-multipath as we are now doing it
with "multipath -i -u", thus the remaining single-path devices would be
classified as non-multipath, and eventually be processed by systemd.
Obvious drawback: single-path devices, more often than not the local
SAS or SATA disks, would be processed very late in the boot process.
This could be worked around by blacklisting, just like now.

I actually have an implementation of this already, but we decided
against it for SUSE. If you're interested, I can post it here as PoC.

Martin

-- 
Dr. Martin Wilck <mwilck at suse.com>, Tel. +49 (0)911 74053 2107
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)