Copy with SCP - low performance

Jean-Eric Cuendet jec at rptec.ch
Tue Mar 30 10:28:38 UTC 2004


> Horacio Reyes wrote:
>
>  >>- The files are copied in batch of 100 tif images, in a directory with
>  >>more than 5 million files.
>  >>- the server that initiate the scp is a 1.2Ghz P3 with 2G in ram, the
>  >>receiver is a double Xeon 2.6ghz with 2G in ram. The first server takes
>  >>no time to send the file, but the receiver takes more than 200segs to
>  >>copy. They are in a 100mb lan.
>  >>- the top comand shows 98% free cpu before the scp, when scp start it
>  >>takes 100% of the 2 xeon cpu. The problem is not the ram, both servers
>  >>have more than 1G free when scp starts.
>  >>
> 
> I have found thet SCP does not like lots of small files, if you can tar
> the files together into on big file it will probably transfer much 
> quicker..

Do you know a command to tar through SSH which is the same as scp but 
kepps better permissions and is quicker?
Here it is:

You have bar/foo and you'd like to copy dir foo to server blabla:
   cd bar && tar c foo | ssh user at blabla "cd /dir_on_blabla && tar x"

Great isn't it?
-jec





More information about the fedora-list mailing list