duplicating a disk

[Kim Lux]

I wrote this about 2 years ago.  I think it is still relevant.  One of
the ways I would improve it is to use a USB-IDE enclosure for the target
drive as you wouldn't have to use Linux rescue nor reboot. 

FWIW : 


I just spent about 2 days figuring out how to easily clone a Linux HD.
I'm
writing this to hopefully save someone else the grief that I've gone
through.

I'm a relative newbie to Linux.  I'm sure the experienced Linux people
can
enhance/correct/improve what I've written here, but at least this works.

My setup is a PC clone with multiple HD bays and a CDROM player.  My
source
drive is a 6.4 GB IDE hard drive with 3 partitions: boot, swap and root.
It is
nearly full.  I'm running RH8.0 using kernel 2.4.18.something.

The new drive, herein called the "clone", is a 60 GB IDE hard drive.

The major steps to cloning a HD are as follows:
1) partition the clone drive
2) format each partition on the clone drive
3) copy files from the old drive to the clone drive
4) set up grub on the clone drive
5) boot it.

The whole process sounds intimidating, but it really isn't any different
or
harder than cloning a DOS drive.  Some people say that one should just
install a
fresh copy of Linux on the new drive, making it a new install rather
than a
clone install, but that is MUCH more work than just making a clone.  My
old
drive has numerous patches, updates, installs and data installed just
about
everywhere. It works exactly as I want it to.  I wouldn't want to repeat
the
work that it took to achieve that, so  I'll clone it rather than start
afresh
with a new drive.

We use removable hard drives on our PCs.  For absolute data safety, when
partitioning and formatting hard drives, I remove all drives except the
one that
I am working on and run from a boot disk.  One could easily perform all
this
work with the new drive mounted alongside an existing drive, but I
don't.  Thus,
all my commands are based on the clone drive being installed as /hda and
running from linux rescue.  If you are performing your clone with the
clone
drive installed alongside an existing drive, your commands will be based
on
/dev/hdb or /dev/hdc, for example.

I also assume that the reader knows how to set up his/her BIOS so that
it has
the proper boot order when needed, ie boot from CDROM first so that
linux rescue
can be run.


In detail:

1) Partition the clone drive.

I partitioned the clone drive by running fdisk from linux rescue booted
from the
install CDROM. When running linux rescue, I DO NOT let the OS mount the
hard
drive.  I always do that manually from the command line when I am doing
system
maintenance.

Once at the command prompt in linux rescue:

#/sbin/fdisk /dev/hda

Obviously you will change this to suit your partitions and partition
types.
REMEMBER THAT I INSTALL THE CLONE DRIVE AS PRIMARY MASTER IE: /dev/hda.
IF YOU
DON'T DO THIS, YOU NEED TO CHANGE THE DRIVE SPECIFICATION

This will start fdisk. From here you are on your own.  I created 3 linux
partitions: boot, swap and root.

I made boot (/dev/hda1) to be 100 MB in size and used ext3 for the file
system.

I made the swap partition (/dev/hda2) twice as big as my motherboard
RAM, or
512MB x 2 = 1 GB.  It used the Linux swap file type.

I made the root (/dev/hda2) partition consume the rest of the drive and
gave it
a ext3 file type as well.

NOTE: One should set the label of the partitions while using fdisk.  I
didn't
and had to later.  The root drive (/dev/hda3), for example, should be
given a
label of "/" to properly work with grub as we'll see later on.



2) Format the partitions

If you are following my drive layout, these are the commands you will
use.

a) Format the boot partition as ext3:
#/sbin/mkfs.ext3 /dev/hda1

b) Format the swap partition as linux swap:
#/sbin/mkfs.swap /dev/hda2

c) Format the root partition as ext3:
#/sbin/mkfs.ext3 /dev/hda3

Obviously you will change this to suit your partitions and partition
types.
REMEMBER THAT I INSTALL THE CLONE DRIVE AS PRIMARY MASTER IE: /dev/hda.
IF YOU
DON'T DO THIS, YOU NEED TO CHANGE THE DRIVE SPECIFICATION


3) Copy the data from the old drive to the clone drive.

To do this, I reboot my computer with both the source and the clone
drive in the
bays.  I usually boot with the source drive in /hda (ie primary master)
and the
clone drive in /hdc (ie secondary master).  My reasoning for doing this
is that
sometimes new drives (ie the clone) have much newer technology than the
old
drive and the two will not operate properly on the same ide channel.  By
putting
one on the primary channel and the other on the secondary channel, I can
clone
really old drives (1995 vintage 1 GB, for example) onto new drives
(latest 120
GB) without any problems.

There are two partions that we need to copy data from: the boot partion
and the
root partion.  I prefer to do these copies as two distinct mounts and
copies.
One could mount the source drive hierarchically and also the clone drive
and
then issue one copy command, but for only 2 partitions I prefer to do 2
separate
copy commands.

Again I boot linux rescue from the CDROM and bypass any automatic
mounting of
filesystems.

First I mount the old boot partion as /oldboot

# cd /
# mkdir oldboot
# mount -text3 /dev/hda1 /oldboot
# cd oldboot
# ls
<you should get a listing of your old boot partition here>

Next, I mount the new boot partition as /newboot

#cd /
# mkdir newboot
# mount -text3 /dev/hdc1 /newboot
# cd newboot
# ls
<you should get a listing of nothing here because it should be empty>

Now, I copy the data from the old boot partition to the new boot
partition:

# cp -aR /oldboot/* /newboot 2>/newboot/error.log

The -a attribute tells cp to keep all of the current attributes.
The -R attribute tells cp to do a recursive copy.
optional: if you want to watch the filenames stream by while they are
being
copied, add a "-v" to the attributes, ie "-avR"

The "2>error.log" tells the OS to redirect the stderror output of cp
into the
error.log file in /newboot.

The boot partition is fairly small, so this shouldn't take very long.
When the
copy is done, I do the following:

# cd /newboot
# ls
<a display of the new partition should occur.  It shouldn't be empty>
# more error.log
< a display of the error.log file should appear.  It should be empty or
nearly
so .>

Note: one could write a script that compared the attributes of every
file in
/newboot to every file in /oldboot, but I haven't found this to be
necessary.
As long as there are no alarming errors in error.log, the new partition
should
have all the files that the source drive had.

Now I repeat that copy process for the root partition:
# cd /
# mkdir newroot
# mount -text3 /dev/hdc3 /newroot
# ls /newroot
<should be empty>

# mkdir oldroot
# mount -text3 /dev/hda3 /oldroot
# ls /oldroot
# <should contain root stuff>

# cp -aR /oldroot/* /newroot 2>/newroot/error.log
# cd newroot
# ls
<a dislpay of the new root should be here>
# more error.log

If both error.log files are clean, consider your copy successful.
Although the
partitions and formated and the data has been copied, the clone drive is
still
not bootable.

BTW: many experienced linux people will tell you to use dd or tar or
several
other copy processes together with pipes and redirection.  They might be
faster,
but I had trouble with most of them in one or more situations. cp is
easy to
understand and I've yet to find an instance where it doesn't work.

4) Set up grub on the new drive.

Here is where I got hung up and where most people get stuck.

First, we need to install grub into the boot sector of the boot
drive/partition.
This does NOT occur automatically when formating the boot partion, nor
when
copying the root files.  This is akin to performing a >sys c: in Dos. To
install
grub on the boot sector, we need to run something called grub-install
from the
command line, but first we must mount our filesystem and set up a new
root
directory using our new filesystem.  Here is how I do that:

a) boot linux rescue with the clone drive installed as /hda ie primary
master.
(It could actually be done with the drive mounted anywhere.)

b) mount the new cloned filesystem as it will be in the computer when
running,
but mount it under /new.  NOTE: some people will tell you to mount it
under
/mnt, which already exists in linux rescue.  Don't do this.  Why ?
Because the
command shell for linux rescue appears to be running under /mnt and
when/if you
mount the new filesystem under /mnt, you will no longer have a shell to
run
under !  Ie, you won't have access to ls, mount, chroot, etc. So, here
are my
commands:

# mkdir new
# mount -text3 /dev/hda3 /new
# ls /new
<listing should occur here>
# mount -text3 /dev/hda1 /new/boot
# ls /new/boot
<listing of /boot should occur here>

We've now got our cloned filesystem mounted exactly as it will be in our
new
system, but under /new.  The boot partition (/dev/hda1) is mounted
in /boot just
like it will be when we run the drive.  We didn't really need to do the
root hierarchical mounting, but I like it that way.

Now, we change the root directory of our filesystem and start a new bash
shell.
We need to do this because grub-install is not present on the linux
rescue
shell:

# chroot /new /bin/bash

This command will make / be what /new/ was.  It will also
start /bin/bash, which
was /new/bin/bash, which is the bash shell on our cloned drive.  We can
now go
to  work using all the tools present on our cloned drive, ie all the
tools that
were present on the old drive.  We can actually startX is we need to,
although I never have. 

We need to install grub onto the boot sector of the boot drive, so:

# /sbin/grub-install /dve/hda1

BTW: if we want to learn about grub, we can use man grub or info grub
now.  info
grup-install is interesting to read.  Neither man nor info are available
in linux rescue mode. 

Grub is now installed on the boot sector, but sometimes it needs to be
configured to find grub.conf.  Grub.conf is the configuration file that
grub
runs to know what kernels to offer for booting, where the images are,
etc.  Grub
config is located in hda1/grub/, which is really /boot/grub. (Remember
that we
mounted hda1 as /new/boot/, but our chroot changed that to /boot/.)

"info grub" in the FAQ part contains this nice little gem:

<paste begins>
I have a separate boot partition and GRUB doesn't recognize it.
     This is often reported as a "bug", but this is not a bug really.
     This is a feature.

     Because GRUB is a boot loader and it normally runs under no
     operating system, it doesn't know where a partition is mounted
     under your operating systems. So, if you have the partition
     `/boot' and you install GRUB images into the directory
     `/boot/grub', GRUB recognizes that the images lies under the
     directory `/grub' but not `/boot/grub'. That's fine, since there
     is no guarantee that all of your operating systems mount the same
     partition as `/boot'.

     There are several solutions for this situation.

       1. Install GRUB into the directory `/boot/boot/grub' instead of
          `/boot/grub'. This may sound ugly but should work fine.

       2. Create a symbolic link before installing GRUB, like `cd /boot
          && ln -s . boot'. This works only if the filesystem of the
          boot partition supports symbolic links and GRUB supports the
          feature as well.

       3. Install GRUB with the command `install', to specify the paths
          of GRUB images explicitly. Here is an example:

               grub> root (hd0,1)
               grub> install /grub/stage1 d (hd0) /grub/stage2
p /grub/grub.conf
<paste ends>

I like option number 3, so that is what I run.  However, my boot drive
is 
/dev/hda1, which is (hd0,0).  (See info grub for info on how grub
specifies 
drives.  Remember that all drive numbers in grub are base 0, not base
1).  I 
thus run the following commands:

# grub
grub> root (hd0,0)
grub> install /grub/stage1 d (hd0) /grub/stage2 p /grub/grub.conf
<if you've got the directories and drive number right, there won't be an
error. 
If one is wrong, there will be a file not found error>
grub> exit

Grub itself is now set up in the boot sector and it knows where
grub.conf is.
If  grub.conf worked well in the source drive, it should work well on
the cloned
drive.

At this point, your drive is bootable, although it probably won't boot.
Here is
the rub: many of the entries in grub, automount and init files refer to
drives
by their LABEL.  Yes, there are now 3 ways to refer to drives in Linux:

1) by the device ie: /dev/hda1, hda2, etc.  see man mount for more on
this.
2) by the grub specification ie: (hd0,0)  see info grub for more on
this.
3) by the drive label.

It is handy for certain linux processes to be able to refer to drives by
their
function (ie root, boot, etc.) rather than have them hardcoded into
the /dev/
spec. Thus, certain processes refer to drives by their label, which is
chosen to
indicate their function.  It turns out that certain processes in the
boot
process  refer to the root drive by its label, which they expect to be
"/".  In
order for our clone drive to boot properly, we need to set its label to
"/".
Issue the following command to do this:

# e2label /dev/hda3 "/"

There might be other ways to do this, like during fdisk or mkfs, but I
am a
newbie, remember.  I think that labels can be checked with hdparm, but I
can't
remember.

Interestingly enough the clone drive boots with only the label for root
changed.
I did not have to change the label for the boot partition.  However, I'm
sure
that somewhere there is an application or process that will refer to the
boot
partition by its label, thus I would set its label to /boot as well:

# el2label /dev/hda1 "/boot"

Your clone drive should now be an exact image of the source drive and
bootable.

Using Clones

I keep a small, outdated drive with nothing but a clean, uptodate copy
of Linux
on it.  Anytime we need to build a new drive for a new computer or a new
process
 (in an existing computer via installing a new drive), I just clone the
"clean"
drive.  This works really well with new users (friends) because I know
how
everything is set up and they don't have to muddle through the
installation
process.

The Beauty of Linux

This whole process could actually be done as one script, with the grub
commands
put into a separate grup scrip file.  I haven't done it yet, but the
whole
cloning process would just be a script that could be run at some period
of
convenience, like just before leaving work for home or just before going
to bed.

I removed the source drive while formatting and partitioning and
rebooted
/remounted when setting up grub, but that wouldn't have to be done that
way.

One more thing: the cloning script could run the new drive and add a few
default
users as well as initiate the email client.

Enjoy !!!


possible changes: use mirrordir, but it isn't available on most machines
without 
installing an RPM

Change the cp attributes.  cp /dir/* won't copy the dot files in /dir/

and this:

<paste>
You are a bit wrong with the cp arguments:
the -a arguments archives a whole directory structure, which means:
 - preserves uig/gid/timestamps,
 - archives recursively
 - Doesn't go through symbolic links (it just copies them as well)

>From the ls manpage, -a is the same with -dpR
So, to mirror your harddisk, just:
cp -avx / /mnt/newdisk

-v: To view what is being done.
-x: To stay to the current filesystem.

<end of paste>

Lastly, mkdir /proc

-- 
Kim Lux,  Diesel Research Inc.