[Linux-cluster] using GNBD versus iSCSI

[Maciej Bogucki]

Benjamin Marzinski napisał(a):
> On Tue, Mar 27, 2007 at 10:22:01PM +0200, David Shwatrz wrote:
>>    Hello,
>>    In short: what are the advantages/disadvantages when using GNBD versus
>>    iSCSI for exporting storage in a cluster?
> 
> The only real advantage of using GNBD is that it has built in fencing. With
> iSCSI, you still need some somthing to fence all the machines (unless your scsi
> target supports SCSI-3 persistent resrvations). Theoretically, GNBD could
> run faster, since it doesn't need to do the work to imitate a SCSI device, but
> but there's a lot of work that needs to be done for GNBD to reach it's speed
> potential. Since there isn't much active development of GNBD, if iSCSI
> isn't already faster than it, it will eventually be. I am pretty much the only
> GNBD developer, and aside from bug fixing, I do very little work on it. iSCSI
> has an active community of developers. Using iSCSI also allows a much more
> seemless transition to a hardware shared storage solution later on.
> 
> If you don't have any fencing hardware, and your iSCSI target doesn't support
> SCSI-3 persistent reservations, then you should probably go with GNBD. Otherwise
> it's up to you.

Hello,

Disadventage of using GNBD is that it doesn't support
authorization/authentication, so it could be a problem in some
configurations.

Here is comparison test with or without GNBD

hardware:
- kernel: 2.6.18-8.1.1
- Procesor: 2x Intel(R) Xeon(TM) CPU 2.80GHz
- Memory: 2GB
- SCSI DISC: MAXTOR    Model: ATLAS10K5_146SCA  Rev: JNZ6
- Filesystem: ext3

1. local test(/dev/sdb1)

[root at blade03-1 ~]# bonnie++ -d /mnt -s 4g -u root
Version  1.03       ------Sequential Output------ --Sequential Input-
--Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
blade03-1        4G 35829  86 93034  43 35334  13 46666  93 84543  11
484.2   1
                    ------Sequential Create------ --------Random
Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
/sec %CP
                 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
+++++ +++
blade03-1,4G,35829,86,93034,43,35334,13,46666,93,84543,11,484.2,1,16,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++

2. GNBD test(/dev/gnbd/sdb1)

- 1Gbit network(Nortel Layer 2/3 Copper GbE Switch Module for BladeCenter)

[root at blade02-1 ~]# bonnie++ -d /mnt -s 4g -u root
Version  1.03       ------Sequential Output------ --Sequential
Input---Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
blade02-1        4G 19724  50 20362  11 13391   7 31688  70 79717  16
360.0 0
                    ------Sequential Create------ --------Random
Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
/sec %CP
                 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
+++++ +++
blade02-1,4G,19724,50,20362,11,13391,7,31688,70,79717,16,360.0,0,16,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++
[root at blade02-1 ~]#

Best Regards
Maciej Bogucki