[linux-lvm] Data corruption on large, multi-device filesystem

[Dan Pritts]

Hi folks - I'm new to the list and to the Linux LVM.

I believe I've just run into the same problem described in this thread from 
back in January. 

x86_64 dell, redhat AS 4, kernel 2.6.9 (redhat source, self-compiled to
add xfs) with two ~5TB RAIDs that I'm striping with LVM.  
(it's really the same RAID presenting two LUNs over two parallel fibre
channel busses.)

About 4TB into a "dd if=/dev/zero of=testfile" the filesystem (XFS) started
complaining bitterly of errors.

On the other hand I can write a full 5TB of zero into an XFS created on
one of the raw devices with no errors.

Both XFS & the dm code in the kernel seem to have had some non-trivial updates
between 2.6.9 from redhat and 2.6.11.7, so i'm compiling that now.

Is it believed that this problem should be fixed in the mainline kernel
or are there other LVM patches I should apply?

Also, seems to be a FAQ on this list but my search of the archives didn't
show any definitive answer - is it true that there are no large device
limits with LVM2/device mapper?  

thanks
danno

On Thu, Jan 20, 2005 at 06:38:41PM -0800, joe at eiler.net wrote:
> Thanks, I'll give it(the patch you sent) a try and let you know what I see.
> 
> When you say "chunk size" what lvcreate arg does that map to? the stripe
> size(-I) or the chunk size(-c) that goes along with snapshots?
> 
> Joe
> 
> Quoting Alasdair G Kergon <agk at redhat.com>:
> 
> > On Wed, Jan 19, 2005 at 02:14:43PM -0800, joe at eiler.net wrote:
> > > I don't see any corruption on the files unless I specify the -i option to
> > > lvcreate.
> >
> > That's a useful observation: different code gets run when -i is used, and
> > I've taken a look and there are two 32-bit limits in that code.
> >
> > One appears to be intentional for performance reasons and can be worked
> > around by increasing the chunk size (default 64K), though it really should
> > detect when the limit is exceeded and tell you!
> >
> > The other unintentional limit (which is the one affecting you I think)
> > leads to wrap-around at striped device offsets beyond ~2^32 sectors
> > i.e. 2^39 bytes into a PV, or ~2^40 bytes (2TB) into an LV in your
> > case of 2 stripes.  The fix for that is a basic exercise in C.
> > [Ref.  stripe_map() in drivers/md/dm-stripe.c]
> >
> > Alasdair
> > --
> > agk at redhat.com
> >
> > _______________________________________________
> > linux-lvm mailing list
> > linux-lvm at redhat.com
> > https://www.redhat.com/mailman/listinfo/linux-lvm
> > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
> >
> 
> 


danno
--
dan pritts - systems administrator - internet2
734/352-4953 office        734/834-7224 mobile