How to optimize the cloning of a disk with a big empty ext3 partition?

[Alex Bligh]

--On 19 June 2009 16:42:54 -0700 Gregoire Gentil <gregoire at gentil.com> 
wrote:

> I need to duplicate a huge number of identical 8GB SD cards on which I
> have 1GB of data at the beginning of the disk on various partitions and
> then a 7GB ext3 partition which is rather empty (just a few MB of data).
>
> The duplicator device enables me to select which sectors I can
> binary-duplicate. I would love to divide by 8 my duplication time, by
> duplicating only the first GB and then the beginning of the 7GB ext3
> partition.

I am guessing here about your application, but how about:

1. Leave the 7GB partition unformatted in the SD card image, set the
   partition type in the partition table to a magic value
2. Make a gzip'd version of a disk image of that partition. Do this by
   using a loopback file system on a file the size of the partition
   which is initially full of a single magic byte/word. The resultant
   file should be tiny as the image will be mostly blocks full of the
   magic byte/word. Put this on another partition.
3. In the boot code for the device (I am assuming you can change that
   or better still it is on the CF device), put a little bit of code
   that
   a) checks the partition table for the magic number, and if set,
      do this:
   b) mounts the partition with the gzip image, extracts it,
      writes it (only writing blocks that are not full of the magic
      byte/word, so the process will be quick) to the new partition,
      then unmounts the partition with the gzip image so the boot
      process can continue
   c) change the partition table entry to ext3
   d) continue the boot process
   Of course this only happens on the first boot, and is only writing
   a few Mb of data.
4. Only duplicate the first 1GB of the disk

The program described is a few tens of lines of C linked with zlib.
It also gives you a relatively easy way to restore the partition
if you should ever need to (change partition table magic number back,
reboot). If this is the only r/w partition, that might be useful.

-- 
Alex Bligh