[Linux-cluster] Iozone tests on gfs with 29th march gfs snapshot
Hansjoerg Maurer
Hansjoerg.Maurer at dlr.de
Wed Apr 13 06:32:24 UTC 2005
Hi,
we are planning a small cluster (3 nodes in the SAN and 13 nodes with gnbd)
and I have tried gfs 6.1 from 29th march on RHEL testkernel
2.6.9-6.36.ELsmp x64 (elevator=deadline) Opteron x64 Hardware
(we want to use RHEL4 based systems, because our cluster application
performes much better with
RHEL4 gcc on the opteron Hardware compared to RHEL3 gcc)
Installation was fine :-)
We have done some iozone runs on a local disk (SAN hardware is not
avaliable yet) with
- gfs and lock_dlm
- gfs mounted with localcache
- ext3
- gnbd (test on remote host)
- nfs (test on remote host)
The avaliable memory of the computers was reduced to 1G to speed up the
test.
There are some interesting points:
- gfs seems to perform nearly as good as ext3 only with reclen 1024 and
during write
- gfs read performance seems to be not very good (are there any flags to
improve it?)
- there seems to be no big difference in mount-option with localcache
and using lock_dlm
- gnbd's write performance seems to be better as nfs
- nfs read performance seems to be better as gnbd's
- running two gnbd tests in parallel reduces read performance
dramatically (may be an hardware issue,
because it seems to be the same with nfs)
We want to use he cluster filesystem mostly to read and proceed 2 GB
datasets.
So we will test it with an application soon, which ist not a synthetic
benchmark.
Is there any prefered elevator, one should use with gfs?
(I will try some tests this evening)
It would be nice, if anyone with gfs experience could comment on the
results.
We will have the hardware avaliable for testing until the mid of next week,
though if someone wants me to try some other configurations (including
current CVS)
give me a note.
Thank you very much
Hansjörg
-- Ext3
Run began: Tue Apr 12 20:55:54 2005
Using minimum file size of 2097152 kilobytes.
Using maximum file size of 2097152 kilobytes.
Auto Mode
Command line used: /opt/iozone/bin/iozone -n 2G -g 2G -a -i 0 -i
1 -i 2
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random
random bkwd record stride
KB reclen write rewrite read reread read
write read rewrite read fwrite frewrite fread freread
2097152 64 56657 55876 38324 37653 11437
20542
2097152 128 59898 56835 38341 38075 19130
33323
2097152 256 61116 57787 35269 38075 15809
43095
2097152 512 61501 58274 37937 38266 23262
39051
2097152 1024 58998 53493 37869 35665 37163
38920
2097152 2048 58333 58310 35291 38273 47074
38147
2097152 4096 60143 60270 37873 38445 54552
37152
2097152 8192 57725 50131 37767 38112 54441
37164
2097152 16384 53178 57389 38317 38283 58545
34425
iozone test complete.
-- GFS lock_dlm
Run began: Tue Apr 12 19:16:51 2005
Using minimum file size of 2097152 kilobytes.
Using maximum file size of 2097152 kilobytes.
Auto Mode
Command line used: /opt/iozone/bin/iozone -n 2G -g 2G -a -i 0 -i
1 -i 2
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random
random bkwd record stride
KB reclen write rewrite read reread read
write read rewrite read fwrite frewrite fread freread
2097152 64 37142 37320 24185 25787 7972
22459
2097152 128 34475 34459 25381 25408 13556
21645
2097152 256 33592 33405 25220 25222 11011
21268
2097152 512 32269 31527 25081 25308 14364
23356
2097152 1024 57865 35774 24190 24919 23027
28430
2097152 2048 41923 34880 25364 24216 29515
31278
2097152 4096 46328 35647 24187 25127 32752
30084
2097152 8192 32296 34830 25172 24000 35714
31385
2097152 16384 40136 38008 24386 25317 39407
37255
iozone test complete.
-- GFS mounted with localcache
Run began: Tue Apr 12 19:17:53 2005
Using minimum file size of 2097152 kilobytes.
Using maximum file size of 2097152 kilobytes.
Auto Mode
Command line used: /opt/iozone/bin/iozone -n 2G -g 2G -a -i 0 -i
1 -i 2
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random
random bkwd record stride
KB reclen write rewrite read reread read
write read rewrite read fwrite frewrite fread freread
2097152 64 35995 35988 24151 26080 8119
21277
2097152 128 31582 35475 25624 25818 13604
22245
2097152 256 30484 34576 25344 25073 11117
21686
2097152 512 29355 35323 25542 25107 14942
24594
2097152 1024 33213 32260 24896 25344 22427
26454
2097152 2048 34852 36949 24417 25377 30143
31454
2097152 4096 42722 33431 24978 24774 32416
31650
2097152 8192 43942 32786 25752 24461 36606
33237
2097152 16384 32568 33575 25072 25506 38057 32971
- NFS mounted ext3
Run began: Tue Apr 12 22:08:04 2005
Include close in write timing
Using minimum file size of 2097152 kilobytes.
Using maximum file size of 2097152 kilobytes.
Auto Mode
Command line used: /opt/iozone/bin/iozone -c -n 2G -g 2G -a -i 0
-i 1 -i 2
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random
random bkwd record stride
KB reclen write rewrite read reread read
write read rewrite read fwrite frewrite fread freread
2097152 64 29933 24934 34428 34329 11396
5095
2097152 128 29848 24387 33693 33815 18280
5822
2097152 256 30033 25856 33915 33520 24666
7303
2097152 512 29868 26192 33932 32201 16207
7797
2097152 1024 30857 25485 31165 32378 27212
9730
2097152 2048 28617 24478 33258 35049 41341
10497
2097152 4096 29514 25804 33221 31459 49188
9050
2097152 8192 30777 25721 32443 32264 46874
8502
2097152 16384 28470 25419 34607 34286 57369
8056
iozone test complete.
- GNBD mounted GFS
Run began: Tue Apr 12 19:16:51 2005
Using minimum file size of 2097152 kilobytes.
Using maximum file size of 2097152 kilobytes.
Auto Mode
Command line used: /opt/iozone/bin/iozone -n 2G -g 2G -a -i 0 -i
1 -i 2
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random
random bkwd record stride
KB reclen write rewrite read reread read
write read rewrite read fwrite frewrite fread freread
2097152 64 37142 37320 24185 25787 7972
22459
2097152 128 34475 34459 25381 25408 13556
21645
2097152 256 33592 33405 25220 25222 11011
21268
2097152 512 32269 31527 25081 25308 14364
23356
2097152 1024 57865 35774 24190 24919 23027
28430
2097152 2048 41923 34880 25364 24216 29515
31278
2097152 4096 46328 35647 24187 25127 32752
30084
2097152 8192 32296 34830 25172 24000 35714
31385
2097152 16384 40136 38008 24386 25317 39407
37255
iozone test complete.
--GNBD mounted GFS (2 simultanous runs und 2 gnbd clients)
Run began: Tue Apr 12 22:33:15 2005
Using minimum file size of 2097152 kilobytes.
Using maximum file size of 2097152 kilobytes.
Auto Mode
Command line used: /opt/iozone/bin/iozone -n 2G -g 2G -a -i 0 -i
1 -i 2
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random
random bkwd record stride
KB reclen write rewrite read reread read
write read rewrite read fwrite frewrite fread freread
2097152 64 21310 20580 9275 10662 3472
10022
2097152 128 11953 17971 8171 10598 4633
17555
2097152 256 15856 22844 5131 11313 4157
25773
2097152 512 15887 28725 4182 12117 7046
27678
2097152 1024 28444 26170 3780 10233 9771
30745
2097152 2048 38449 27667 4042 10059 12872
30426
2097152 4096 27096 35431 4699 11047 15139
31802
2097152 8192 29305 37275 4696 9728 14856
39259
2097152 16384 26824 49085 8836 5279 16865
57405
iozone test complete.
--
_________________________________________________________________
Dr. Hansjoerg Maurer | LAN- & System-Manager
|
Deutsches Zentrum | DLR Oberpfaffenhofen
f. Luft- und Raumfahrt e.V. |
Institut f. Robotik |
Postfach 1116 | Muenchner Strasse 20
82230 Wessling | 82234 Wessling
Germany |
|
Tel: 08153/28-2431 | E-mail: Hansjoerg.Maurer at dlr.de
Fax: 08153/28-1134 | WWW: http://www.robotic.dlr.de/
__________________________________________________________________
There are 10 types of people in this world,
those who understand binary and those who don't.
More information about the Linux-cluster
mailing list