Copy with SCP - low performance
Jean-Eric Cuendet
jec at rptec.ch
Tue Mar 30 10:28:38 UTC 2004
> Horacio Reyes wrote:
>
> >>- The files are copied in batch of 100 tif images, in a directory with
> >>more than 5 million files.
> >>- the server that initiate the scp is a 1.2Ghz P3 with 2G in ram, the
> >>receiver is a double Xeon 2.6ghz with 2G in ram. The first server takes
> >>no time to send the file, but the receiver takes more than 200segs to
> >>copy. They are in a 100mb lan.
> >>- the top comand shows 98% free cpu before the scp, when scp start it
> >>takes 100% of the 2 xeon cpu. The problem is not the ram, both servers
> >>have more than 1G free when scp starts.
> >>
>
> I have found thet SCP does not like lots of small files, if you can tar
> the files together into on big file it will probably transfer much
> quicker..
Do you know a command to tar through SSH which is the same as scp but
kepps better permissions and is quicker?
Here it is:
You have bar/foo and you'd like to copy dir foo to server blabla:
cd bar && tar c foo | ssh user at blabla "cd /dir_on_blabla && tar x"
Great isn't it?
-jec
More information about the fedora-list
mailing list