[dm-devel] 'multipath add path' returns ok but the path is missing in the topology

[Gioh Kim]

Hi Martin,

On Thu, Apr 23, 2020 at 3:58 PM Martin Wilck <mwilck at suse.com> wrote:
>
> On Thu, 2020-04-23 at 13:50 +0200, Gioh Kim wrote:
> > Hi Martin,
> >
> > On Wed, Apr 22, 2020 at 5:40 PM Martin Wilck <mwilck at suse.com> wrote:
> > > On Wed, 2020-04-22 at 16:00 +0200, Gioh Kim wrote:
> > > > One question.
> > > > Is it possible for multipathd to generate an event for removing a
> > > > map?
> > > > I am looking into the source code, ev_add_path and some functions
> > > > related to add a path,
> > > > but it looks like it does not generate any event.
> > >
> > > I guess you mean an uevent. These events are created by the kernel
> > > when
> > > multipath creates or reloads maps. multipathd normally doesn't
> > > trigger
> > > the events directly.  But there are some cases where multipathd
> > > basically runs the equivalent of "udevadm trigger -c change" on
> > > either
> > > maps or paths. Search the code for "trigger" for details.
> > >
> > > There are also device mapper events which are created by the kernel
> > > for
> > > certain state changes of dm devices. Again, multipathd only
> > > receives
> > > such events.
> > >
> > > Regards
> > > Martin
> > >
> > >
> > >
> >
> > I am making an environment to capture log for uevent as you said.
> >
> > Meanwhile I found something weird: "orphan path, waiting for create
> > to
> > complete" what is this message?
> > I guess the second added path was not registered when I executed
> > "multipathd add path" at this moment.
>
> True. "orphan" means that you added a path that didn't belong to a map.
>
> Again, your multipathd seems to be configured such that maps aren't
> created automatically ("find_multipaths strict"). I'm not sure whether
> that makes a lot of sense in your setup, the way you describe it.
> With "find_multipaths smart" or "find_multipaths greedy", your new maps
> would be added without admin intervention.

Yes, exactly.
I disabled all automatic work.
Actually it is legacy code. I do not know why.
As like all other legacy code, it is not easy to change and test.

I am really sorry to not inform you of the version of multipath-tools.
It's 0.6.4 from Debian stretch repository.

>
> >
> > I guess there would be some time delay for the second path to be
> > included in the topology.
> > Am I right?
> > If I am right, could I make "multipathd add path" return only after
> > everything is completely finished?
>
> What is "everything"? how would multipathd know that you intend to add
> another path? As I said previousl, you'd probably be better off using
> "multipath -a $WWID", which would add the new WWID to the WWIDs file,
> which controls multipathd's operation in "find_multipaths strict" mode.
>
> > Apr 23 11:38:25 ib1 multipathd[2132]: sdc: open node: /dev/sdc
> > Apr 23 11:38:25 ib1 multipathd[2132]: sdc: domap with mpp->action=6
> > Apr 23 11:38:25 ib1 multipathd[2132]:
> > 360000000000000000000000000000000: load table [0 204800 multipath 1
>
> This WWID looks very fishy to me. Especially when you add or remove
> paths dynamically, be sure to use unique WWIDs.

They are test devices with fixed WWIDs.

>
> Regards
> Martin
>
>

I would like to inform you of what I found out.
1. 'multipathd add path sdx' is executed
2. Adding thd sdx also creates a map.
3. The multipathd daemon waits for the change uevent from kernel.
4. Another sd device sdy is added with 'multipathd add path sdy' command
5. adding sdy is not done and multipathd prints a message "sdy: orphan
path, waiting for create to complete"
6. Something (I guess it is the checker thread) removes and adds the dm device
7. sdy is removed with a message "sdy: orphan path, path removed
externally". I guess the checker thread removes sdy as well.

I am not sure what removes the dm device and sdy device.
I guess it would be the check thread due to the "sdy: orphan path,
path removed externally" message.

Therefore I just added the 'udevadm settle' command after the
'multipathd add path sdx' command.
So whenever my script executes 'multipathd add path', it also executes
'udevadm settle'.
Then my test is fine for a long time.


-- 
Gioh Kim

Cloud server kernel maintainer
Quality Management (IONOS Cloud)

1&1 IONOS SE | Greifswalder Str. 207 | 10405 Berlin | Germany
Phone: +49 176 26978962
E-mail: gi-oh.kim at cloud.ionos.com | Web: www.ionos.de

Hauptsitz Montabaur, Amtsgericht Montabaur, HRB 24498

Vorstand: Dr. Christian Böing, Hüseyin Dogan, Dr. Martin Endreß,
Hans-Henning Kettler, Arthur Mai, Matthias Steinberg, Achim Weiß
Aufsichtsratsvorsitzender: Markus Kadelke

Member of United Internet

Diese E-Mail kann vertrauliche und/oder gesetzlich geschützte
Informationen enthalten. Wenn Sie nicht der bestimmungsgemäße Adressat
sind oder diese E-Mail irrtümlich erhalten haben, unterrichten Sie
bitte den Absender und vernichten Sie diese E-Mail. Anderen als dem
bestimmungsgemäßen Adressaten ist untersagt, diese E-Mail zu
speichern, weiterzuleiten oder ihren Inhalt auf welche Weise auch
immer zu verwenden.

This e-mail may contain confidential and/or privileged information. If
you are not the intended recipient of this e-mail, you are hereby
notified that saving, distribution or use of the content of this
e-mail in any way is prohibited. If you have received this e-mail in
error, please notify the sender and delete the e-mail.