[linux-lvm] system boot time regression when using lvm2-2.03.05

[Martin Wilck]

On Tue, 2019-09-10 at 22:38 +0200, Zdenek Kabelac wrote:
> Dne 10. 09. 19 v 17:20 David Teigland napsal(a):
> > > > > _pvscan_aa
> > > > >    vgchange_activate
> > > > >     _activate_lvs_in_vg
> > > > >      sync_local_dev_names
> > > > >       fs_unlock
> > > > >        dm_udev_wait <=== this point!
> > > > > ```
> > > Could you explain to us what's happening in this code? IIUC, an
> > > incoming uevent triggers pvscan, which then possibly triggers VG
> > > activation. That in turn would create more uevents. The pvscan
> > > process
> > > then waits for uevents for the tree "root" of the activated LVs
> > > to be
> > > processed.
> > > 
> > > Can't we move this waiting logic out of the uevent handling? It
> > > seems
> > > weird to me that a process that acts on a uevent waits for the
> > > completion of another, later uevent. This is almost guaranteed to
> > > cause
> > > delays during "uevent storms". Is it really necessary?
> > > 
> > > Maybe we could create a separate service that would be
> > > responsible for
> > > waiting for all these outstanding udev cookies?
> > 
> > Peter Rajnoha walked me through the details of this, and explained
> > that a
> > timeout as you describe looks quite possible given default
> > timeouts, and
> > that lvm doesn't really require that udev wait.
> > 
> > So, I pushed out this patch to allow pvscan with --noudevsync:
> > https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=3e5e7fd6c93517278b2451a08f47e16d052babbb
> > 
> > You'll want to add that option to lvm2-pvscan.service; we can
> > hopefully
> > update the service to use that if things look good from testing.
> 
> This is certainly a bug.
> 
> lvm2 surely does need to communication with udev for any activation.
> 
> We can't let running activation 'on-the-fly' without control on
> system with 
> udev (so we do not issue 'remove' while there is still 'add' in
> progress)
> 
> Also any more complex target like thin-pool need to wait till
> metadata LV gets 
> ready for thin-check.

My idea was not to skip synchronization entirely, but to consider
moving it to a separate process / service. I surely don't want to re-
invent lvmetad, but Heming's findings show that it's more efficient to
do activation in a "single swoop" (like lvm2-activation.service) than
with many concurrent pvscan processes.

So instead of activating a VG immediately when it sees all necessary
PVs are detected, pvscan could simply spawn a new service which would
then take care of the activation, and sync with udev.

Just a thought, I lack in-depth knowledge of LVM2 internals to know if
it's possible.

Thanks
Martin