[linux-lvm] Cached root filesystem LV causes excessive disk activity following (re)boot.
zxzorn at yahoo.com
zxzorn at yahoo.com
Sat Jun 12 20:24:26 UTC 2021
I have a system where the root filesystem is contained in a LV that sits on top of a software RAID 1 device that uses two mechanical drives. Also in the VG is a partition on an NVMe device. I have used the LVM tools to create a small cache (dm-cache) for the root filesystem, with the cache on the NVMe device. I'll give the commands I used to create this at the end of this message, in case that information should be helpful.
I'm experiencing a problem when the system is (re)booted: as soon as the initrd script activates the VG that contains the root filesystem, but even before this (or any other) filesystem is mounted, I hear the sound of frantic seeking from the two mechanical drives. This continues while the system boots and all this activity causes the boot process to take several times longer than it did before I created the cache. The excessive disk activity continues for a time once booting is complete and the system is nominally idle. This causes system performance to be very sluggish during this time. This problem does not occur if I remove the cache. It also does not happen if I reboot the system immediately after re-creating the cache (when the cache must be just about empty). I've found that the duration of the excessive disk activity depends on how full the cache LV is, with more data in the cache causing it to go on for longer. With approx. 8 GB in the cache, it takes around 20 minutes for the activity to stop. Clearly, this defeats most of the point of having the cache. The excessive activity seems to be due to a kernel process called kcopyd, but I have no idea what it's doing (presumably copying data from somewhere to somewhere else, with one of those somewheres being the root filesystem LV).
Could it be related to the fact that the VG cannot be deactivated at system shutdown because the root filesystem is still mounted (albeit read only at this point and no other filesystems are mounted)?
I've used a number of recent 5.10.x and 5.12.x versions of the Linux kernel and exactly the same problem has been present in all cases.
Can anyone shed any light on what is happening here and on how it might be avoided?
The cache was added to the LV as follows:
lvcreate -L 15G --name root-cacheddata vg1 /dev/nvme0n1p3
lvcreate -L 16M --name root-cachedmdata vg1 /dev/nvme0n1p3
lvconvert --type cache-pool --cachemode writethrough --poolmetadata vg1/root-cachedmdata vg1/root-cacheddata
lvconvert --type cache --cachepool vg1/root-cacheddata vg1/root (where root is the LV containing the root filesystem)
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the linux-lvm