[linux-lvm] system boot time regression when using lvm2-2.03.05

[Zdenek Kabelac]

Dne 11. 09. 19 v 9:17 Martin Wilck napsal(a):
> On Tue, 2019-09-10 at 22:38 +0200, Zdenek Kabelac wrote:
>> Dne 10. 09. 19 v 17:20 David Teigland napsal(a):
>>>>>> _pvscan_aa
>>>>>>     vgchange_activate
>>>>>>      _activate_lvs_in_vg
>>>>>>       sync_local_dev_names
>>>>>>        fs_unlock
>>>>>>         dm_udev_wait <=== this point!
>>>>>> ```
>>>> Could you explain to us what's happening in this code? IIUC, an
>>>> incoming uevent triggers pvscan, which then possibly triggers VG
>>>> activation. That in turn would create more uevents. The pvscan
>>>> process
>>>> then waits for uevents for the tree "root" of the activated LVs
>>>> to be
>>>> processed.
>>>>
>>>> Can't we move this waiting logic out of the uevent handling? It
>>>> seems
>>>> weird to me that a process that acts on a uevent waits for the
>>>> completion of another, later uevent. This is almost guaranteed to
>>>> cause
>>>> delays during "uevent storms". Is it really necessary?
>>>>
>>>> Maybe we could create a separate service that would be
>>>> responsible for
>>>> waiting for all these outstanding udev cookies?
>>>
>>> Peter Rajnoha walked me through the details of this, and explained
>>> that a
>>> timeout as you describe looks quite possible given default
>>> timeouts, and
>>> that lvm doesn't really require that udev wait.
>>>
>>> So, I pushed out this patch to allow pvscan with --noudevsync:
>>> https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=3e5e7fd6c93517278b2451a08f47e16d052babbb
>>>
>>> You'll want to add that option to lvm2-pvscan.service; we can
>>> hopefully
>>> update the service to use that if things look good from testing.
>>
>> This is certainly a bug.
>>
>> lvm2 surely does need to communication with udev for any activation.
>>
>> We can't let running activation 'on-the-fly' without control on
>> system with
>> udev (so we do not issue 'remove' while there is still 'add' in
>> progress)
>>
>> Also any more complex target like thin-pool need to wait till
>> metadata LV gets
>> ready for thin-check.
> 
> My idea was not to skip synchronization entirely, but to consider
> moving it to a separate process / service. I surely don't want to re-
> invent lvmetad, but Heming's findings show that it's more efficient to
> do activation in a "single swoop" (like lvm2-activation.service) than
> with many concurrent pvscan processes.
> 
> So instead of activating a VG immediately when it sees all necessary
> PVs are detected, pvscan could simply spawn a new service which would
> then take care of the activation, and sync with udev.
> 
> Just a thought, I lack in-depth knowledge of LVM2 internals to know if
> it's possible.


Well for relatively long time we do want to move 'pvscan' back to be processed 
within udev rules  and activation service being really just a service
doing  'vgchange -ay'.

Another floating idea is to move towards monitoring instead of using semaphore
(since those SysV resources are kind-of limited and a bit problematic
when there are left in the system).


Regards

Zdenek