<html><head>
<meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body
text="#000000" bgcolor="#FFFFFF">
Hello,<br>
<br>
I'm having a problem uncaching logical volumes when the cache data
chunck size is over 1MiB.<br>
The process I'm using to uncache is: <span style="font-style: italic;">lvconvert
--uncache vg/lv<br>
<br>
<br>
</span>The issue occurs across multiple systems with different hardware
and different versions of LVM.<br>
<br>
Steps to reproduce:<br>
<ol>
<li>Create origin VG & LV<br>
</li>
<li>Add cache device over 1TB to the origin VG</li>
<li>Create the cache data lv: <span style="font-style: italic;"><br>
lvcreate -n cachedata -L 1770GB cached_vg /dev/nvme0n1</span></li>
<li><span style="font-style: italic;"></span>Create the cache metadata
lv: <br>
<span style="font-style: italic;">lvcreate -n cachemeta -L 1770MB
cached_vg /dev/nvme0n1</span></li>
<li><span style="font-style: italic;"></span>Convert to a cache pool: <br>
<span style="font-style: italic;">lvconvert --type cache-pool
--cachemode writethrough --poolmetadata cached_vg/cachemeta
cached_vg/cachedata</span></li>
<li>Enable caching on the origin LVM: <br>
<span style="font-style: italic;">lvconvert --type cache --cachepool
cached_vg/cachedata cached_vg/filestore01</span></li>
<li>Write some data to the main LV so as the cache device is used: <span
style="font-style: italic;"><br>
dd if=/dev/zero of=/mnt/filestore01/test.dat bs=1M count=10000</span></li>
<li>Check the cache stats: <br>
<span style="font-style: italic;">lvs -a -o
+cache_total_blocks,cache_used_blocks,cache_dirty_blocks</span></li>
<li>Repeating step 8 over time will show that the dirty blocks are not
being written back at all</li>
<li>Try to uncache the device:<br>
<span style="font-style: italic;">lvconvert --uncache cached_vg/</span><span
style="font-style: italic;">filestore01</span></li>
<li><span style="font-style: italic;"></span>You will get a repeating
message. This will loop indefinitely and not decrease or complete:<br>
<span style="font-style: italic;">Flushing x blocks for cache
cached_vg/filestore01.</span>
</li>
</ol>After testing multiple times, the issue seems to be tied to the
chunk size selected in step 5. The LVM man page mentions that the chunk
must be a multiple of 32KiB, however the next chunk size automatically
assigned over 1MiB is usually 1.03MiB. With a chunk size of 1.03MiB or
higher, the cache is not able to flush. Creating a cache device with a
chunk size of 1MiB or less, the cache is flushable.<br>
<br>
Now knowing how to avoid the issue, I just need to be able to safely
un-cache systems with do have a cache that will not flush.<br>
<p>Details:<br>
</p>Version info from <span style="font-style: italic;">lvm version</span>:<br>
<p>
LVM version: 2.02.171(2)-RHEL7 (2017-05-03)<br>
Library version: 1.02.140-RHEL7 (2017-05-03)<br>
Driver version: 4.35.0<br>
Configuration: ./configure
--build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu
--program-prefix= --disable-dependency-tracking --prefix=/usr
--exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin
--sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include
--libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var
--sharedstatedir=/var/lib --mandir=/usr/share/man
--infodir=/usr/share/info --with-default-dm-run-dir=/run
--with-default-run-dir=/run/lvm --with-default-pid-dir=/run
--with-default-locking-dir=/run/lock/lvm --with-usrlibdir=/usr/lib64
--enable-lvm1_fallback --enable-fsadm --with-pool=internal
--enable-write_install --with-user= --with-group= --with-device-uid=0
--with-device-gid=6 --with-device-mode=0660 --enable-pkgconfig
--enable-applib --enable-cmdlib --enable-dmeventd --enable-blkid_wiping
--enable-python2-bindings --with-cluster=internal --with-clvmd=corosync
--enable-cmirrord --with-udevdir=/usr/lib/udev/rules.d
--enable-udev_sync --with-thin=internal --enable-lvmetad
--with-cache=internal --enable-lvmpolld --enable-lvmlockd-dlm
--enable-lvmlockd-sanlock --enable-dmfilemapd<br>
</p>
<p>System info:<br>
System 1,2,3:<br>
- Dell R730XD server<br>
- 12x disk in RAID 6 to onboard PERC/Megaraid controller<br>
</p>
<p>System 4:<br>
-Dell R630 server<br>
-60x Disk (6 luns) in RAID 6 to PCI megaraid controller<br>
</p>
<p>The systems are currently in production, so it's quite hard for me
to change the configuration to enable logging.<br>
</p>
<p> Any assistance would be much appreciated! If any more info is
needed please let me know.<br>
Best regards,<br>
Ryan<br>
</p>
<span style="color: rgb(80, 80, 80);"></span>
</body>
</html>