[Linux-cluster] GFS, how many nodes do you have?

[Marcelo Matus]

I saw some benchmarks over 100 Mbs/ ethernet, they got around *5MB/s 
(single  IDE disk?).*

https://open.datacore.ch/DCwiki.open/Wiki.jsp?page=GFS.Bonnie

    Version  1.03       - - - Sequential Output - - - --Sequential Input- --Random-
                        -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
    Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
    System2-GNBD-GFS 1G  4190  27  7185  11  2777   5  4165  26  5029   4  96.4   3
                        - - - Sequential Create - - - - - - - Random Create - - - -
                        -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
                  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                     16   262  38 12996  35   147  11   203  31   557   7   150  12

*

We use Infiniband and we are getting around 35MB/s (SCSI raid 5/SGI 
TP9100).*

Bonnie 1.4: File './Bonnie.8320', size: 1048576000, volumes: 1
Writing with putc()...         done:  34619 kB/s  90.6 %CPU
Rewriting...                   done: 459206 kB/s  99.6 %CPU
Writing intelligently...       done: 213785 kB/s  41.5 %CPU
Reading with getc()...         done:  35544 kB/s 100.0 %CPU
Reading intelligently...       done: 1066698 kB/s 100.0 %CPU

I guess you should get a similar performance over Gigabit.

For comparison, our Infiniband network provides only around 2 times the 
bandwith over TCP/IP
(1800 Mb/s) than our GB network (800Mb/s).

Marcelo


Jacob Joseph wrote:

> Hi.  I run a very similar dual opteron cluster here at CMU and would 
> be very interested in any responses you might have recieved off of the 
> list.  Additionally, do you happen to know of any benchmarks of gnbd 
> performance over gigabit?
>
> Thanks,
> -Jacob
>
> Marcelo Matus wrote:
>
>> Dear GFS list members:
>>
>> We are planning to install GFS in our new Dual Opteron cluster
>> (20 nodes, ie 40cpus) and probably to serve some workstation
>> (another 10-20), considering one GFS server and using gndb.
>>
>> I have seen in the list that there were some problems for more
>> than 10 nodes, but now that is fixed.
>>
>> Still, the question is, does someone has experience with
>> ~40 clients + 1 GFS server?, is that too much?
>>
>> Marcelo
>>
>>
>>
>>
>> -- 
>> Linux-cluster mailing list
>> Linux-cluster at redhat.com
>> http://www.redhat.com/mailman/listinfo/linux-cluster
>
>
> -- 
> Linux-cluster mailing list
> Linux-cluster at redhat.com
> http://www.redhat.com/mailman/listinfo/linux-cluster