[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: copying large files between filesystems



On Tue, 2004-06-29 at 18:43 -0700, Andrew Scott wrote:
> Hello, this isn't necessarily a Fedora specific question either.  It is 
> happening while operating from a Fedora Core II install, and I just 
> subscribed to this mailing list which seems to get a good deal of 
> talented traffic.

Certainly a good deal of traffic! ;^)

> Please allow me to throw this one out there and I 
> promise to be of assistance with other people's problems in the future.  :)
> 
> I'm trying to copy a large bzip2 archive that is a little bit larger 
> than 2 Gigs, from a disk with a resierfs filesystem on it, to another 
> disk with ext3 filesystem.  I only mention the filesystem types because 
> I think this might be the cause of my problem.

IIRC older resiserfs (version < 3.6) had a 2GB limit, so it depends.
Seems like it should have given an error.  The limit on ext3 file size
depends on the kernel, so again it depends on the details of your
system.  Particular tools may also have built-in filesize limits - have
been some discussions about getting large files such ad DVD images with
tools such as wget.  Archive tools may have the same issues.

> Immediately after I made 
> the original bzip2 archive I ran md5sum on the resulting file, and kept 
> a copy of that number.

Not quite clear on the archive issue - bzip2 is a compression method,
not an archive tool.  What kind of archive is it?  Can you get a
directory of the original archive in place?  (e.g. For a tar archive,
could you do "tar jtvf archive_name.tar.bz"?

> After I copy the file to another filesystem, I 
> can not get the same md5sum.  I've tried cp, rsync, scp, and dd and they 
> each seem to come up with different md5sums.  The byte count is the same 
> however, but the md5sum is always different, so something however 
> minimal is changing and henceforth throwing off the whole bzip2 archive 
> and rendering it unexstractable.

If you can read the directory in place, you may be able to extract to
the new target location (at least up to the 2GB limit if not beyond)
without moving the archive.

(e.g. "tar jxvf archive_name.tar.bz -C /new/filesystem/location")

> I'm totally stressed because this is a 
> back up of my homedir from before a reinstall.  :-/  And I'm worried 
> that the original filesystem that I wrote the backup to (the reiserfs 
> filesystem) was so old that it didn't recognize files greater than 2 
> Gigs, silently.
> 
> If I md5sum the file, still in it's place, it's still good.

But if it's already effectively truncated at 2GB it may not actually be
good.

> 
> Any ideas why I can't get matching md5sums from a file copy between 
> filesystems of different type?  Any ideas how to recover from this?
> 
> Thanks in advance, anyone, with thoughts,

Provide more details and you may be able to get better help.

How To Ask Questions The Smart Way
http://www.catb.org/~esr/faqs/smart-questions.html

Phil





[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]