[Linux-cluster] Slow tar

[Kadlecsik Jozsi]

Hi,

We have been testing GFS over AoE (over Gb Ethernet) and in our last test 
what we tried is to untar a 38GB file with 150000 files. The process was 
started two days ago and it still hasn't completed. We see that it 
proceeds in creating the new files and directories, but this seems to be 
way too slow.

This is a non-tuned 5-node GFS cluster with four nodes alive. On each 
node:

# uname -r
2.6.20.4-grsec,
# gfs_tool version | head -1
gfs_tool DEVEL.1177069643 (built Apr 20 2007 13:48:40)
# cman_tool status | head -1
Protocol version: 5.0.1

AoE and GFS use a dedicated LAN segment with MTU set to 9216.

What makes it more strange is that previously we ran bonnie++ which 
produced the following results:

Local ext3: bonnie++ -d /usr/src/test -u nobody

Version  1.03       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
atlas           12G 39019  96 50354  22 23123   6 34311  69 49428   5 207.6   1
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  2747  94 +++++ +++ +++++ +++  2910  99 +++++ +++  9811 100

GFS: bonnie++ -d /export/home/test -u nobody

Version  1.03       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
atlas           12G 29911  86 57432  33 16625  13 39512  96 17334   4 310.4   0
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  1455  37 +++++ +++  2958  31  1549  43 +++++ +++  4496  48

and that suggested better "real life" performance compared to what we got
at running a huge tar.

What can be wrong here? How could we improve the performance?

Best regards,
Jozsef
--
E-mail : kadlec at sunserv.kfki.hu, kadlec at blackhole.kfki.hu
PGP key: http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address: KFKI Research Institute for Particle and Nuclear Physics
         H-1525 Budapest 114, POB. 49, Hungary