[linux-lvm] Why is the performance of my lvmthin snapshot so poor

[Zhiyong Ye]

Hi Gionatan,

Thanks for your reply and detailed answer.

I actually use iSCSI for my underlying storage and the bare disk random 
write iops is 65543. Some of the parameters of my iSCSI initiator are as 
follows:
node.session.iscsi.FirstBurstLength = 524288
node.session.iscsi.MaxBurstLength = 33552384
node.session.cmds_max = 4096
node.session.queue_depth = 1024

After creating the PV and VG based on the iSCSI device, I created the 
thin pool as follows:
lvcreate -n pool -L 1000G test-vg
lvcreate -n poolmeta -L 100G test-vg
lvconvert --type thin-pool --chunksize 64k --poolmetadata 
test-vg/poolmeta test-vg/pool
lvchange -Z n test-vg/pool

Then I create thin lv in the thin pool:
lvcreate -n test-thin -V 500G --thinpool pool test-vg

And my command for creating snapshots:
lvcreate -n test-thin1s1 -s test-vg/test-thin

I have the following fio parameter and use it for all tests:
[global]
bs=4k
direct=1
iodepth=32
numjobs=8
ioengine=libaio
group_reporting
runtime=120
time_based
filename=/dev/vdb

[rand-write]
name=rand-write
rw=randwrite
stonewall

Thanks again!

Zhiyong Ye

在 6/14/22 3:04 PM, Gionatan Danti 写道:
> Il 2022-06-13 10:49 Zhiyong Ye ha scritto:
>> The performance degradation after snapshotting is expected as writing
>> to a snapshotted lv involving reading the original data, writing it
>> elsewhere and then writing new data into the original chunk. But the
>> performance loss was so much more than I expected. Is there any way to
>> improve performance after creating a snapshot? Can I ask for your
>> help?
> 
> This is the key point: when first writing to a new chunk, not only it 
> needs to be allocated, but old data must be copied. This r/m/w operation 
> transform an async operation (write) on a sync one (read), ruining 
> performance. Subsequent writes to the same chunk does have the same issue.
> 
> The magnitute of the slowdown seems somewhat excessive, though. When 
> dealing with HDD pools, I remember a 3-5x impact on IOPs. Can you show 
> the exact fio command and the parameters of your thin pool (ie: chunk 
> size) and storage subsystem (HDD vs SSD, SATA vs SAS vs NVME)?
> 
> Regards.
>