[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Ext4 and large (>8TB) files

Ok, so I guess ext4 with 64-bit, or another filesystem for us. Thanks Andreas & Eric for all the help.

On Fri, Mar 26, 2010 at 3:38 PM, Andreas Dilger <adilger dilger ca> wrote:
On 2010-03-26, at 16:05, Arun Nair wrote:
So I'm guessing switching the system to 64-bit would fix this for us. How about increasing the block size from the current 4k? Would that be an option too?

Not in the near future, unless you are running on PPC/ARM/SPARC that can also handle large pages.

On Fri, Mar 26, 2010 at 2:10 PM, Eric Sandeen <sandeen redhat com> wrote:
On 03/26/2010 03:50 PM, Arun Nair wrote:
> Eric,
> Thanks for the quick reply... see my responses inline...
> On Fri, Mar 26, 2010 at 12:16 PM, Eric Sandeen <sandeen redhat com
> <mailto:sandeen redhat com>> wrote:
>     On 03/26/2010 01:52 PM, Arun Nair wrote:
>     > Hi -
>     >
>     > (I apologize for the ext4 question in an ext3 mailer, but I couldn't
>     > find a user list for ext4.)
>     linux-ext4 vger kernel org <mailto:linux-ext4 vger kernel org> :)
>      but that's ok.
> Saw that but thought it was a dev-only list, sorry. Next time :)

*shrug* I think user questions are welcome too.  At least I don't mind.


> dd fails as mentioned above. xfs_io errors too:
> [root camanoe]# xfs_io -F -f -c "pwrite 8T 1M" bigfile2
> pwrite64: File too large

Oh.  Well, then!  Must be something else.

oh, ok:

sbi->s_bitmap_maxbytes = ext4_max_bitmap_size();

static loff_t ext4_max_size(int blkbits, int has_huge_files)
      loff_t res;
      loff_t upper_limit = MAX_LFS_FILESIZE;


      /* Sanity check against vm- & vfs- imposed limits */
      if (res > upper_limit)
              res = upper_limit;

      return res;


/* Page cache limit. The filesystems should put that into their s_maxbytes
 limits, otherwise bad things can happen in VM. */

so it's only giving us 31 bits of pages, not 32.  This limits it to 8T
on a 32-bit machine with 4k pages.

I'm not honestly sure if there is anything in the vm that can't actually
cope with a 32-bit offset... but until proven otherwise, probably not
going to change this without a lot of testing & inspection.


Ext3-users mailing list
Ext3-users redhat com

Cheers, Andreas

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]