Ext4 and large (>8TB) files

[Arun Nair]

Ok, so I guess ext4 with 64-bit, or another filesystem for us. Thanks
Andreas & Eric for all the help.

On Fri, Mar 26, 2010 at 3:38 PM, Andreas Dilger <adilger at dilger.ca> wrote:

> On 2010-03-26, at 16:05, Arun Nair wrote:
>
>> So I'm guessing switching the system to 64-bit would fix this for us. How
>> about increasing the block size from the current 4k? Would that be an option
>> too?
>>
>
> Not in the near future, unless you are running on PPC/ARM/SPARC that can
> also handle large pages.
>
>  On Fri, Mar 26, 2010 at 2:10 PM, Eric Sandeen <sandeen at redhat.com> wrote:
>> On 03/26/2010 03:50 PM, Arun Nair wrote:
>> > Eric,
>> >
>> > Thanks for the quick reply... see my responses inline...
>> >
>> > On Fri, Mar 26, 2010 at 12:16 PM, Eric Sandeen <sandeen at redhat.com
>> > <mailto:sandeen at redhat.com>> wrote:
>> >
>> >     On 03/26/2010 01:52 PM, Arun Nair wrote:
>> >     > Hi -
>> >     >
>> >     > (I apologize for the ext4 question in an ext3 mailer, but I
>> couldn't
>> >     > find a user list for ext4.)
>> >
>> >     linux-ext4 at vger.kernel.org <mailto:linux-ext4 at vger.kernel.org> :)
>> >      but that's ok.
>> >
>> >
>> > Saw that but thought it was a dev-only list, sorry. Next time :)
>>
>> *shrug* I think user questions are welcome too.  At least I don't mind.
>>
>> ...
>>
>> > dd fails as mentioned above. xfs_io errors too:
>> > [root at camanoe]# xfs_io -F -f -c "pwrite 8T 1M" bigfile2
>> > pwrite64: File too large
>>
>> Oh.  Well, then!  Must be something else.
>>
>> oh, ok:
>>
>> sbi->s_bitmap_maxbytes = ext4_max_bitmap_size();
>>
>> static loff_t ext4_max_size(int blkbits, int has_huge_files)
>> {
>>       loff_t res;
>>       loff_t upper_limit = MAX_LFS_FILESIZE;
>>
>> <snip>
>>
>>       /* Sanity check against vm- & vfs- imposed limits */
>>       if (res > upper_limit)
>>               res = upper_limit;
>>
>>       return res;
>> }
>>
>> and:
>>
>> /* Page cache limit. The filesystems should put that into their s_maxbytes
>>  limits, otherwise bad things can happen in VM. */
>> #if BITS_PER_LONG==32
>> #define MAX_LFS_FILESIZE (((u64)PAGE_CACHE_SIZE << (BITS_PER_LONG-1))-1)
>>
>> so it's only giving us 31 bits of pages, not 32.  This limits it to 8T
>> on a 32-bit machine with 4k pages.
>>
>> I'm not honestly sure if there is anything in the vm that can't actually
>> cope with a 32-bit offset... but until proven otherwise, probably not
>> going to change this without a lot of testing & inspection.
>>
>> -Eric
>>
>> _______________________________________________
>> Ext3-users mailing list
>> Ext3-users at redhat.com
>> https://www.redhat.com/mailman/listinfo/ext3-users
>>
>
>
> Cheers, Andreas
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/ext3-users/attachments/20100326/59eba361/attachment.htm>