[linux-lvm] Discussion: performance issue on event activation mode

[heming.zhao at suse.com]

On 6/17/21 12:38 AM, David Teigland wrote:
> On Thu, Jun 17, 2021 at 12:18:47AM +0800, heming.zhao at suse.com wrote:
>> I don't know if I missed something, the result didn't show any progress.
>> the result of "devices/obtain_device_list_from_udev=0" even got regression: from 23.3 => 39.8
>>
>> the lvm2 version with dev-dct-device-info-1 branch code
>> ```
>> sle15sp2-base40g:~ # lvm version
>>    LVM version:     2.03.13(2)-git (2021-05-07)
>>    Library version: 1.02.179-git (2021-05-07)
>>    Driver version:  4.40.0
>>    Configuration:   ./configure --host=x86_64-suse-linux-gnu --build=x86_64-suse-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/lib --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --disable-dependency-tracking --enable-dmeventd --enable-cmdlib --enable-udev_rules --enable-udev_sync --with-udev-prefix=/usr/ --enable-selinux --enable-pkgconfig --with-usrlibdir=/usr/lib64 --with-usrsbindir=/usr/sbin --with-default-dm-run-dir=/run --with-tmpfilesdir=/usr/lib/tmpfiles.d --with-thin=internal --with-device-gid=6 --with-device-mode=0640 --with-device-uid=0 --with-dmeventd-path=/usr/sbin/dmeventd --with-thin-check=/usr/sbin/thin_check --with-thin-dump=/usr/sbin/thin_dump --with-thin-repair=/usr/sbin/thin_repair --enable-blkid_wiping --enable-lvmpolld --enable-realtime --with-cache=internal --with-default-locking-dir=/run/lock/lvm --with-default-pid-dir=/run --with-default-run-dir=/run/lvm --enable-fsadm --disable-silent-rules --enable-write_install --with-vdo=none
>> ```
>>
>> Installation with cmd: make install && dracut -f --add "lvm"
>>
>> top 10 blame services
>> ```
>> external_device_info_source = "none"
>> obtain_device_list_from_udev = 1
>> event_activation = 1
>> udev_sync = 1
> 
> Thanks for running that again.  From your previous testing, my conclusion
> was that libudev caused the slowdown.  So, the patch is supposed to avoid
> all libudev calls by default, and rely only on lvm's native device type
> detection.  The default settings to avoid libudev, are:
> 
>    obtain_device_list_from_udev = 0
>    external_device_info_source = "none"
> 
> I'm not sure which of your test results match those settings, but if those
> results are not improved, then we should look further to see if the
> slowdown is caused by something other than libudev calls.
> 

the default value of external_device_info_source is "none" and I didn't change it.
So below (from my last mail) is your wanted result, it's worse than before (23.3 vs 39.8).

```
devices/obtain_device_list_from_udev=0
global/event_activation=1
activation/udev_sync=1

sle15sp2-base40g:~ # systemd-analyze blame | head -n 10
          39.845s lvm2-pvscan at 133:576.service
          39.830s lvm2-pvscan at 133:640.service
          39.829s lvm2-pvscan at 133:720.service
          39.827s lvm2-pvscan at 132:736.service
          39.825s lvm2-pvscan at 132:656.service
          39.823s lvm2-pvscan at 132:672.service
          39.821s lvm2-pvscan at 132:720.service
          39.820s lvm2-pvscan at 132:544.service
          39.819s lvm2-pvscan at 132:624.service
          39.808s lvm2-pvscan at 132:576.servic
```

the booting time out problem cause by two side:
1> libudev take much time
2> thousands of lvm2-pvscan at .service running on the same time

the new patch or existing code (with obtain_device_list_from_udev=0) can avoid <1>.
but <2> is still not to avoid. (direct activation can avoid it)

Thanks,
heming