cpio to ext4 seems much slower than to ext2, ext3 or xfs
Ric Wheeler
rwheeler at redhat.com
Thu Nov 12 20:52:34 UTC 2009
On 11/12/2009 03:27 PM, Eric Sandeen wrote:
> Roberto Ragusa wrote:
>> Ric Wheeler wrote:
>>> In our testing with f12, I build a 60TB ext4 file system with 1 billion
>>> small files. A forced fsck of ext4 finished in 2.5 hours give or take a
>>> bit :-) The fill was artificial and the file system was not aged, so
>>> real world results will probably be slower.
>>>
>>> fsck time scales mostly with the number of allocated files in my
>>> experience. Allocated blocks (fewer very large files) are quite quick.
>>>
>>
>> What kind of machine did you use?
>>
>> With 60TB a simple allocation bitmap for 4k-blocks takes almost 2GB;
>> and this is just to detect free space or double allocation of blocks.
>> Wow.
>>
>
> The box did have a lot of memory, it's true :)
>
> But ext4 also uses the "uninit_bg" feature:
>
> uninit_bg
> Create a filesystem without initializing all of the
> block groups. This feature also enables checksums
> and highest-inode-used statistics in each block-
> group. This feature can speed up filesystem cre-
> ation time noticeably (if lazy_itable_init is
> enabled), and can also reduce e2fsck time dramati-
> cally. It is only supported by the ext4 filesystem
> in recent Linux kernels.
>
> -Eric
>
A lot in this case was 40GB of DRAM - fsck (iirc) consumed about 13GB of virtual
space during the run?
Ric
More information about the fedora-devel-list
mailing list