[Linux-cluster] GNBD speed
Chris Picton
lists at tangent.co.za
Mon Apr 21 14:00:04 UTC 2008
Hi All
I am testing the following scenario:
A DRBD mirror between two servers, which heartbeat failover the drbd
primary, gnbd export and ip address.
I am trying to find potential bottlenecks, and have done the following
tests.
Network speed between the DRBD servers (A and B)
---------------------------------------------------------
(A) dd if=/dev/zero bs=1G count=1 | nc 10.100.1.2 5001
(B) nc k -l 5001 | dd of=/dev/null
(A) reports:
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 7.6384 seconds, 141 MB/s
DRBD sync speed:
----------------------------------------------------------
dd if=/dev/zero bs=1G count=1 of=/dev/drbd0 oflag=sync
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 10.7832 seconds, 99.6 MB/s
Network speed between GNBD export (A) and import (C)
-----------------------------------------------------------
(C) dd if=/dev/zero bs=1G count=1 | nc nfs1 5001
(A) nc -k -l 5001 | dd of=/dev/null
(C) reports:
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 10.4001 seconds, 103 MB/s
Network speed between GNBD import (C) and export (A)
-----------------------------------------------------------
(A) dd if=/dev/zero bs=1G count=1 | nc 10.200.3.10 5001
(C) nc -k -l 5001 | dd of=/dev/null
(A) reports:
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 10.4001 seconds, 93 MB/s
So I have established that writing to drbd directly is fast, and network
speed is fast
However, using gnbd as follows:
on the drbd server:
gnbd_serv -n
/sbin/gnbd_export -c -e r0 -d /dev/drbd0
On the client:
gnbd_import -i 10.200.3.3
I try the speed tests over the gnbd devices:
Reading from GNBD:
------------------------------------------------------------
dd if=/dev/gnbd0 of=/dev/null bs=1G count=1
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 17.0842 seconds, 62.8 MB/s
Writing to GNBD (no sync flag)
------------------------------------------------------------
dd if=/dev/zero of=/dev/gnbd0 bs=1G count=1
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 54.4142 seconds, 19.7 MB/s
Writing to GNBD (sync flag)
------------------------------------------------------------
dd if=/dev/zero of=/dev/gnbd0 bs=1G count=1 oflag=sync
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 53.3085 seconds, 20.1 MB/s
I am almost happy with the 62 Mb/s read speed, but the 20 MB/sec write
speed seems a bit low, compared to the write rate to drbd, and the
network speed.
Can anyone give any hints for optimising the gnbd write speed (and read
speed)
Chris
More information about the Linux-cluster
mailing list