[linux-lvm] device-mapper may have read performance issue

[Peter Keller]

Hi,

On Fri, 10 Oct 2008, Eugene Vilensky wrote:

> Hi all,
> How important are these read ahead settings for random, database IO?

Good question: by coincidence I've just had to exercise a system that has LVM2 
volumes set up on a hardware RAID-6 array (SATA-II disks and an Adaptec 
ICP5085BL card). I took the opportunity to do some bonnie++ tests with readahead 
settings of 1024 and 8192. I was running 5 instances of bonnie++ at the same 
time (each on its own LVM volume, but locked together with the -p/-y options). I 
got numbers like these:

Readahead 1024:

                     ------Sequential Create------ --------Random Create--------
                     -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
server1 (buildho 16  3946  99 +++++ +++  2957 100  4125  99 +++++ +++ 11400  88

Readahead 8192:

                     ------Sequential Create------ --------Random Create--------
                     -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files:max:min        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
server1 (buildho 16  3968  99 +++++ +++  5934 100  4113  99 +++++ +++  5037  99

So, on this system readahead makes no difference for create, and read is so 
quick that bonnie++ doesn't even want to report any numbers. However, sequential 
delete is about twice as fast with the larger readahead, and random delete is 
twice as fast with the smaller readahead.

It is hard to tune for general usage as opposed to specific tasks, and by making 
some things better you may make other things worse. I responded to the original 
question on the assumption that doing a sequential dd represented something like 
the expected usage pattern for the disks.

On the basis of this paticular (very limited) test on this particular system, I 
would answer Eugene's question by saying that the readahead setting is equally 
important for random and sequential I/O, but in opposite directions :-)

YMMV, especially on a typical desktop system where the disks are connected 
directly to the motherboard or via an eSATA port, and if RAID is being used at 
all it is likely to be software RAID-0 or RAID-1

Regards,
Peter.

>
> On 10/10/08, Ben Huang <ben_devel at yahoo.cn> wrote:
>> Hi all,
>>
>> --- Peter Keller <pkeller at globalphasing.com>写道:
>>
>>> Hi all,
>>>
>>> On Wed, 8 Oct 2008, thomas62186218 at aol.com wrote:
>>>
>>>> Ben,
>>>>
>>>> I have seen this same issue as well. I have
>>> created an md device capable of
>>>> 425MB/sec using the hdparm -t command, yet an LVM
>>> volume fully comprising
>>>> this md device only got about 150MB/sec. I am not
>>> sure what the issue is. I
>>>> am running Ubuntu Hardy 804 server edition,
>>> 64-bit.
>>>>
>>>> -Thomas
>>>
>>> I have fixed this kind of problem by tweaking the
>>> readahead of the LVM
>>> volume using 'blockdev --setra' and/or 'blockdev
>>> --setfra'.
>>
>> It make effect
>>
>>
>> blockdev --setra 65536 /dev/md0
>> blockdev --setra 65536 /dev/mapper/DG5-lv1
>>
>> dd if=/dev/md0 of=/dev/null bs=1M
>> 353185562624 bytes (353 GB) copied, 420.033 s, 841
>> MB/s
>>
>> dd if=/dev/mapper/DG5-lv1 of=/dev/null bs=1M
>> 155551006720 bytes (156 GB) copied, 216.576 s, 718
>> MB/s
>>
>> Warm regards,
>> -Ben
>>
>>
>>       ___________________________________________________________
>>  雅虎邮箱，您的终生邮箱！
>> http://cn.mail.yahoo.com/
>>
>> _______________________________________________
>> linux-lvm mailing list
>> linux-lvm at redhat.com
>> https://www.redhat.com/mailman/listinfo/linux-lvm
>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>>
>
> -- 
> Sent from my mobile device
>
> Eugene Vilensky
> evilensky at gmail.com
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>

--
Peter Keller                                     Tel.: +44 (0)1223 353033
Global Phasing Ltd.,                             Fax.: +44 (0)1223 366889
Sheraton House,
Castle Park,
Cambridge CB3 0AX
United Kingdom