[dm-devel] dm-thin vs lvm performance

[Jagan Reddy]

Hi,
I  recently started using dm-thin module and first of all, thank you for the good work. It works great, well documented and the comments in the code are very useful.  I tried to run some performance tests using a high performance flash device and the performance of dm thin is about 30% of LVM performance on a full allocated thin provisioned volume. ( ie after all pages/blocks are allocated using dd). Performance test does only reads and no writes. 

To reproduce the scenario easily, I tried same operations using ramdisk and dd. 

To read 3.2G of data using dd, 

   Raw Ramdisk takes - 2.5 seconds
   LVM Lun - takes 4 seconds
   TP Lun ( after fully allocated) - takes 18 seconds



Steps involved
------------------
1. Linux 3.2.0.rc4 version from kernel.org
 3.2.0-rc4 #1 SMP Fri Dec 16 13:25:02 EST 2011 x86_64 x86_64 x86_64 GNU/Linux
2. LVM2.2.02.88
3. compiling and installing dm thin modules.
4. Create a ramdisk of size 4G.

Ramdisk Raw Read Test
--------------------------
[root at lab-dm-cn4 ~]# dd of=/dev/null if=/dev/ram bs=4096 count=786432
786432+0 records in
786432+0 records out
3221225472 bytes (3.2 GB) copied, 2.56607 s, 1.3 GB/s

LVM SETUP
------------------------

Create PV with 8M metadata on ramdisk , chunksize=8M in VG, lv uses 100%VG.

root at lab-dm-cn4 ~]# dd of=/dev/null if=/dev/6C067CWX00132-1/engineering bs=4096 count=$BLOCKS
                                786432+0 records in
                                786432+0 records out
                                3221225472 bytes (3.2 GB) copied, 4.06687 s, 792 MB/s



DM THIN SETUP
-----------------------
Create metadata device of size 8M and data device of size 3G using dm-linear on ramdisk.
Create TP Pool using the above devices with page/block size of 8M.
Create TP LUN in the TP Pool.
Using dd, write 4k to every page in the TP LUN so that all pages are allocated.
using "dmsetup status" verify that all pages in the pool are allocated.

[root at lab-dm-cn4 ~]# dd of=/dev/null if=/dev/mapper/thin1 bs=4096 count=$BLOCKS
                                          786432+0 records in
                                          786432+0 records out
                                          3221225472 bytes (3.2 GB) copied, 18.2166 s, 177 MB/s



I have attached the output file and the script file used for this test.  Any ideas / suggestions on what may be the bottle neck in dm-thin performance is much appreciated.

Thanks,
Jagan.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/dm-devel/attachments/20120112/c901dbf4/attachment.htm>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: dm_thin_lvm_compare_output.txt
URL: <http://listman.redhat.com/archives/dm-devel/attachments/20120112/c901dbf4/attachment.txt>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lvm_thin_compare.sh
Type: application/x-sh
Size: 4216 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/dm-devel/attachments/20120112/c901dbf4/attachment.sh>