rsyncing 79gb of data to 250gb drive

[Michael Mansour]

Hi James,

 --- James Wilkinson <james at westexe.demon.co.uk>
wrote: > I recommended to Michael Mansour:
> > You can also use
> > cd /data01
> > for i in * ; do du -sk "$i" "/data02/$i" ; done |
> more
> 
> Michael replied:
> > Ok, I've tried this and the following is the
> result:
> > 
> > [root at server data01]# for i in * ; do du -sk "$i"
> > "/data02/$i" ; done |more
> > 4       lost+found
> > 12      /data02/lost+found
> > 79596252        Software
> > 123557857       /data02/Software
> > 
> > Which seems to again confirm that the directories
> are
> > exact copies of one another (which is what rsync
> is
> > meant to do), but again with much different sizes.
> 
> Well, in that case, you could always try
> cd /data01/Software
> for i in * ; do du -sk "$i" "/data02/Software/$i" ;
> done | more
> to see which of the directories inside Software have
> increased in size.
> 
> Then you can try similar drill-downs.
> 
> This will show you, basically, whether it's only
> part of the contents
> which have increased requirements (in which case,
> you look more
> closely at that data), or whether the storage
> requirements for
> everything has increased (in which case, there's
> probably something
> odd about one or other filesystem).
> 
> James.

I tried this and the files themselves were the same.

In the interim what I did while testing this stuff was
to buy some ATA133 cables. I then moved the Maxtor
drive from the IDE0 interface (which comes inbuilt on
the MSI motherboard) and put it into the secondary
interface of the Adaptec 1200A IDE raid controller.

Rebooting the drive came up the same as before, but
when running a "fdisk -l /dev/hdf1" it said the
partition table was not valid.

So I decided to blow away the drive again and re-fdisk
it, this time the cylinders recognised on the drive
had changed from 16000 (thereabouts) to 30514
cylinders.

I re-fdisked, reformatted (mke2fs -m 0 -j -b 4096
/dev/hdf1) - notice I changed back up to 4096 block
size instead of the previously formatted 1024 byte
block size. Remounted the drive as /data02.

I then did:

rsync -av -S /data01/ /data02

The -S (or --sparse is the command line switch another
in this list suggested to put on for the sparse files
on rsync).

The following was the result:

/dev/md6              85419328  79678672   1401516 
99%  /data01

/dev/hdf1            241263968  79482904 161781064 
33% /data02

Much better result than before.

Because I have made 2 changes (1) the changing of the
interface from internal IDE0 to the secondary
interface on the Adaptec 1200A controller, (2) the
addition of the -S option on the rsync, I'm not sure
which of the changes made this work correctly.

What I'm about to do now is blow the drive away again,
keeping the same fdisk structure and re-rsyncing
withOUT the -S option and seeing what's the result.

If it turns out with very similar size, then that
means it's the IDE0 interface on the motherboard
causing the problem, maybe being unable to properly
recognise the 250Gb drive or improperly accessing it
when writing. It's a recent MSI board (bought in
November 2003) with 1Gb RAM and dual AMD Athlon 2400+
cpu's, with current BIOS firmware so it would surprise
me if MSI have got it wrong somehow.

Michael.


Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com