rsyncing 79gb of data to 250gb drive

[T. 'Nifty New Hat' Mitchell]

On Wed, Jun 02, 2004 at 11:46:38AM -0600, Rodolfo J. Paiz wrote:
> Date: Wed, 02 Jun 2004 11:46:38 -0600
> To: For users of Fedora Core releases <fedora-list at redhat.com>
> From: "Rodolfo J. Paiz" <rpaiz at simpaticus.com>
> Subject: RE: rsyncing 79gb of data to 250gb drive
> Reply-To: For users of Fedora Core releases <fedora-list at redhat.com>
> 
> At 15:20 5/28/2004, Michael Mansour wrote:
> >Not only do I wonder if tar will respond the same way
> >as rsync without the -S option, but I'm also wondering
> >if backup software which use rsync to perform backups
> >would also take this into account...
> 
> I also wonder if there is any downside to using -S or whether it might 
> produce an undesirable result in some cases. If not, perhaps one should 
> always use it just in case?

Sparse files are a boon or a bust.  

If you make a sparse file 

  open(); write(byte); lseek(way-out-there); write(byte); close()

you have not reserved or allocated the disk blocks for all the
possible stuff in the middle.  If you expect those blocks to be real
and there is not enough disk space in the future things will bust.

In some cases if you compress the file then expand it you can run
out of disk space on the uncompress pass (compress, gzip, bzip2, zip).
Watch out for uncompress bombs.

Some backup and restore tools will run you out of tape or out of disk
or both with sparse files. Gnu tar has a --sparse flag.  You might not
be able to restore the contents of a file system if you are not
cautious/ aware.

Some data base files might be sparse but you could run out of space
a year later when you do not expect and need the blocks to be real.

mkswap (to files) might generate files that look like they should be
sparse to backup and restore tools but should not be. i.e. You should
not use swapon on a file with holes.

Holes on a file system with one size of allocation block may not be
holes on another file system with a smaller allocation block size.
Since allocation block size is  heuristically determined one could
be confused when moving files from one FS to another of a different 
size or type.  mv does not have a --sparse flag that I can see.

The big issue is backup and restore processes (including cp, mv).

I guess a utility (holeify and holecount) might be written.


-- 
	T o m  M i t c h e l l 
	/dev/null the ultimate in secure storage.