[linux-lvm] Discussion: performance issue on event activation mode

Martin Wilck martin.wilck at suse.com
Tue Sep 28 06:34:06 UTC 2021

Hello David and Peter,

On Mon, 2021-09-27 at 10:38 -0500, David Teigland wrote:
> On Mon, Sep 27, 2021 at 12:00:32PM +0200, Peter Rajnoha wrote:
> > > - We could use the new lvm-activate-* services to replace the
> > > activation
> > > generator when lvm.conf event_activation=0.  This would be done by
> > > simply
> > > not creating the event-activation-on file when event_activation=0.
> > 
> > ...the issue I see here is around the systemd-udev-settle:
> Thanks, I have a couple questions about the udev-settle to understand
> that
> better, although it seems we may not need it.
> >   - the setup where lvm-activate-vgs*.service are always there (not
> >     generated only on event_activation=0 as it was before with the
> >     original lvm2-activation-*.service) practically means we always
> >     make a dependency on systemd-udev-settle.service, which we
> > shouldn't
> >     do in case we have event_activation=1.
> Why wouldn't the event_activation=1 case want a dependency on udev-
> settle?

You said it should wait for multipathd, which in turn waits for udev
settle. And indeed it makes some sense. After all: the idea was to
avoid locking issues or general resource starvation during uevent
storms, which typically occur in the coldplug phase, and for which the
completion of "udev settle" is the best available indicator.

> >   - If we want to make sure that we run our "non-event-based
> > activation"
> >     after systemd-udev-settle.service, we also need to use
> >     "After=systemd-udev-settle.service" (the "Wants" will only make
> > the
> >     udev settle service executed, but it doesn't order it with
> > respect
> >     to our activation services, so it can happen in parallel - we
> > want
> >     it to happen after the udev settle).
> So we may not fully benefit from settling unless we use After
> (although
> the benefits are uncertain as mentioned below.)

Side note :You may be aware that the systemd people are deprecating
this service (e.g.
I'm arguing against it (perhaps you want to join in :-), but odds are
that it'll disappear sooner or later. Fot the time being, I don't see a
good alternative.

The dependency type you have to use depends on what you need. Do you
really only depend on udev settle because of multipathd? I don't think
so; even without multipath, thousands of PVs being probed
simultaneously can bring the performance of parallel pvscans down. That
was the original motivation for this discussion, after all. If this is
so, you should use both "Wants" and "After". Otherwise, using only
"After" might be sufficient.

> > Now the question is whether we really need the systemd-udev-settle
> > at
> > all, even for that non-event-based lvm activation. The udev-settle
> > is
> > just to make sure that all the udev processing and udev db content
> > is
> > complete for all triggered devices. But if we're not reading udev
> > db and
> > we're OK that those devices might be open in parallel to lvm
> > activation
> > period (e.g. because there's blkid scan done on disks/PVs), we
> > should be
> > OK even without that settle. However, we're reading some info from
> > udev db,
> > right? (like the multipath component state etc.)
> - Reading the udev db: with the default
> external_device_info_source=none
>   we no longer ask the udev db for any info about devs.  (We now
> follow
>   that setting strictly, and only ask udev when source=udev.)

This is a different discussion, but if you don't ask udev, how do you
determine (reliably, and consistently with other services) whether a
given device will be part of a multipath device or a MD Raid member?

I know well there are arguments both for and against using udev in this
context, but whatever optimizations you implement, they should work
both ways.

> - Concurrent blkid and activation: I can't find an issue with this
>   (couldn't force any interference with some quick tests.)

In the past, there were issues with either pvscan or blkid (or
multipath) failing to open a device while another process had opened it
exclusively. I've never understood all the subtleties. See systemd
commit 3ebdb81 ("udev: serialize/synchronize block device event
handling with file locks").

> - I wonder if After=udev-settle could have an incidental but
> meaningful
>   effect of more PVs being in place before the service runs?

After=udev-settle will make sure that you're past a coldplug uevent
storm during boot. IMO this is the most important part of the equation.
I'd be happy to find a solution for this that doesn't rely on udev
settle, but I don't see any.


> I'll try dropping udev-settle in all cases to see how things look.
> Dave

More information about the linux-lvm mailing list