cpio to ext4 seems much slower than to ext2, ext3 or xfs

[Ric Wheeler]

On 11/12/2009 01:30 PM, Dennis J. wrote:
> On 11/12/2009 05:59 PM, Eric Sandeen wrote:
>> Dennis J. wrote:
>>> On 11/12/2009 04:03 PM, Eric Sandeen wrote:
>>>> Richard W.M. Jones wrote:
>>
>> ...
>>
>>>>> I'd like to repeat my proviso: I think this test is meaningless for
>>>>> most users.
>>>>
>>>> Until users have 8TB raids at home, which is not really that far off
>>>> ...
>>>
>>> Let's hope btrfs is production ready before then because extX doesn't
>>> look like a fitting filesystem for such big drives due their lack of
>>> online fsck.
>>
>> ext4's fsck is much faster than ext3's, and xfs's repair tool is also
>> pretty speedy.
>>
>> Both are offline, but so far online fsck for btrfs is just a goal, no
>> (released, anyway) code yet AFAIK.
>
> Isn't the speed improvement of ext4 achieved by not dealing with empty
> extends/blocks? If so that wouldn't help you much if those 8TB are
> really used. But even a speedy fsck is going to take longer and longer
> as filesystem size grows which is why I believe we will soon reach a
> point were offline-fsck simply isn't a viable option anymore.
> I have a 30TB storage system that I chopped into ten individual volumes
> because current filesystems don't really make creating a single 30TB fs
> a wise choice even though I'd like to be able to do that.
>
> Regards,
> Dennis
>

In our testing with f12, I build a 60TB ext4 file system with 1 billion small 
files. A forced fsck of ext4 finished in 2.5 hours give or take a bit :-) The 
fill was artificial and the file system was not aged, so real world results will 
probably be slower.

fsck time scales mostly with the number of allocated files in my experience. 
Allocated blocks (fewer very large files) are quite quick.

ric