[linux-lvm] (LONG) Delay when writing to LVM after boot

Fri Apr 19 20:32:16 UTC 2013

Performing a block-level write directly to the logical volume could help determine the answer to whether the problem is with the logical volume or the file system.  You have to be willing to destroy your file system though (because you might overwrite important FS structures).  Obviously, the FS should not be mounted.  If you do this, make sure you try writing > 50M to various places in the addressable space, for example:
~> dd if=/dev/zero of=/dev/VG_NAS/LV_NAS bs=4M count=20 seek=102400  # Skip 400GiB and write 80MiB

 brassow

P.S.  Reading data from the volume and then writing it out to the same location could keep you from blowing away your FS data.  The FS would have to be unmounted.  You would have to deactivate/activate between the read and write to eliminate caching possibility.  Don't count on not making any mistakes.

On Apr 19, 2013, at 6:48 AM, Marian Csontos wrote:

> Likely a question for ext4 forum - I guess FS is reading[1] and caching[2] additional metadata to find suitable block.
> 
> [1] explains the delay on first write
> [2] explains no delay on subsequent writes
> 
> -- Marian
> 
> On 04/19/2013 03:16 AM, Ken Bass wrote:
>> I have one LVG with one LV consisting of several physical drives, ranging
>> from 500G to 3T, with a total size of 6.5T, formatted with ext4.
>> 
>> After I boot the system, when anything writes a file to that LV, the write
>> hangs for 1 minute or more, then continues at full speed with no errors.
>> Some apps, though can't handle the delay, and report that the write fails.
>> 
>> This does not occur with a small file ( < 50M or so ). But the first time a
>> larger file is written, then the delay. And this delay occurs only one time.
>> 
>> I have noticed something similar in the past with smaller LVs ( < 1T ), but
>> the delay was insignificant then.  But as I grew this LV, the delay
>> increased.
>> 
>> I can run e2fsck on the LV, and even optimize directories (-D option), with
>> no delay occurring.
>> 
>> I have tried googling, but have not found any mention of something like
>> this (maybe not using the right key words?). So, does this sound familiar
>> to any LVM experts out there? Any suggestions?
>> 
>> FWIW: I'm currently running Fedora 17 (64 bit), kernel
>> 3.8.4-102.fc17.x86_64 (although I have seen this problem on 32 bit
>> kernels/machines as well). I have 4 physical drives (500G, 1T, 1.5T, 3T),
>> and non-raid configured.
>> 
>> Some other info:
>> 
>> [root at elmer ken]# lvmdiskscan
>>   /dev/vg_elmer/lv_swap [       3.62 GiB]
>>   /dev/sda1             [     500.00 MiB]
>>   /dev/vg_elmer/lv_root [      23.81 GiB]
>>   /dev/sda2             [      27.47 GiB] LVM physical volume
>>   /dev/VG_NAS/LV_NAS    [       5.46 TiB]
>>   /dev/sdb1             [       2.73 TiB] LVM physical volume
>>   /dev/sdc1             [     465.76 GiB] LVM physical volume
>>   /dev/sdd1             [     931.51 GiB] LVM physical volume
>>   /dev/sde1             [       1.36 TiB] LVM physical volume
>>   3 disks
>>   1 partition
>>   0 LVM physical volume whole disks
>>   5 LVM physical volumes
>> [root at elmer ken]# lvdisplay
>>   --- Logical volume ---
>>   LV Path                /dev/VG_NAS/LV_NAS
>>   LV Name                LV_NAS
>>   VG Name                VG_NAS
>>   LV UUID                IO93PY-kFF2-19u7-33lX-m7Mg-8BZC-q12ttn
>>   LV Write Access        read/write
>>   LV Creation host, time ,
>>   LV Status              available
>>   # open                 1
>>   LV Size                5.46 TiB
>>   Current LE             1430794
>>   Segments               5
>>   Allocation             inherit
>>   Read ahead sectors     auto
>>   - currently set to     256
>>   Block device           253:2
>> [root at elmer ken]#
>> 
>> (note: sda is separate physical drive with system/kernel only - vg_elmer-*)
>> 
>> T IA
>> 
>> ken
>> 
>> 
>> 
>> _______________________________________________
>> linux-lvm mailing list
>> linux-lvm at redhat.com
>> https://www.redhat.com/mailman/listinfo/linux-lvm
>> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>> 
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/