[linux-lvm] dm-cache / lvmcache - no cleanup for dirty data in writeback/cleaner mode

CoolCold coolthecold at gmail.com
Mon Jun 6 17:33:17 UTC 2022

Good day!

Main issue as I see it: dirty data size grows, cached data size
declines, no automagic flushing happens (or much slower than growing
Sub issue: cleaner policy flushes data back to origin device very slowly.

  LVM version:     2.03.07(2) (2019-11-30)
  Kernel 5.4.0-109-generic #123-Ubuntu SMP Fri Apr 8 09:10:54 UTC 2022
x86_64 x86_64 x86_64 GNU/Linux

Sizes: 2TB origin/700GB cache device
  [cachedata_cpool]       vg0 Cwi---C--- 700.00g
         99.99  21.21           72.70
  [cachedata_cpool_cdata] vg0 Cwi-ao---- 700.00g
  [cachedata_cpool_cmeta] vg0 ewi-ao----  20.00m

Let's start from cleaner mode, as it what is happening on server right now:

root at TasteWP2:~# lvs -o+cache_policy,cache_settings
  LV       VG  Attr       LSize   Pool              Origin       Data%
 Meta%  Move Log Cpy%Sync Convert CachePolicy CacheSettings
  data     vg0 Cwi-aoC---   2.00t [cachedata_cpool] [data_corig] 99.99
 21.21           72.70            cleaner

and Cpy%Sync moving down very slowly - over 4 hours, it moved from
72.74% to 72.70% only.

The origin device (a couple of HDDs in mdadm RAID1) is almost idling,
5-15% busy based on iostat.

graphical representation from collectd/rrd - https://snipboard.io/WNdGhz.jpg

What I've tried to do: play around setting migration_threshhold, but
see no changes
lvchange --cachesettings 'migration_threshold=65536' vg0/data
lvchange --cachesettings 'migration_threshold=8192' vg0/data

I think this subissue leads to the main issue with dirty growing up,
not leaving space for "real" data.

My very limited knowledge, checking through history on github
back to 2019 (time of kernel release)
have not provided any hints on fixes/changes added here, so I'm bit
lost. Help is appreciated.

Best regards,

More information about the linux-lvm mailing list