[Linux-cluster] GFS performance

[Robert Peterson]

Frazier, Darrell USA CRC (Contractor) wrote:
>
> Hello guru's,
>
> Interesting problem I am hoping someone on this forum has seen before, 
> and can give me pointers to what may be wrong. Here is the setup:
>
> 2 HP DL380 G4 systems with RHEL4U4
> 1 unmanaged switch (for Cluster interconnect and Oracle RAC 
> interconnect.)
> Public network for Oracle NET and normal network traffic
> RHCS4 using DLM locking protocol (three servers to keep GULM lock info 
> for a two node RAC seemed pretty wasteful)
> GFS6.1
> Oracle Clusterware (latest)
> Oracle Database 10gR2
> Fiber-Channel shared storage
>
> Here is my issue:
>
> I have set up a two-node Oracle 10gR2 RAC system with RHCS/GFS (OCFS2 
> was such a disappointment), and Oracle Clusterware. Everything on the 
> OS cluster level is good and fine.
>
> Shortly after setting up the cluster and handing it over to the DBAs 
> to install Clusterware and Oracle RAC. They come to me saying that 
> they have uneven performance between the nodes. (Node one does 
> adatabase import in an hour whereas the same import on Node2 takes 10 
> hours)
>
> I have been doing everything I can do on my side using various tools 
> to try to isolate the issue (protocol analysis, iostat, strace, dd to 
> do writes from both nodes to GFS, etc.) and I have been unable to 
> isolate the issue as performance is quite even on an OS level.  
>
> The cluster configuration has been ruled out by Redhat support as an 
> issue. (Yay for me and Redhat!)
>
> I am hoping someone here has run into issues using Oracle Clusterware 
> on RHCS/GFS.
>
> Thanx in advance
>
> *Darrell J. Frazier*
>
Hi Darrell,

Well, it's possible you're running into this:

http://sources.redhat.com/cluster/faq.html#gfs_speed1

To rule that out, perhaps you could do the same sequence of events after 
a complete cluster reboot,
for each node.  It would be interesting to know if the speed factor changes.

It could be a number of other things, too, including hardware.  You 
could try swapping cables and
ports on the Ethernet and also the shared storage.  By the way, I'm in 
the process of adding a new
"GFS performance tuning" question to the cluster FAQ that may or may not 
help.  It should appear
in the FAQ soon, pending review by some of the developers here (I don't 
want to post any
misinformation, so I'm having them review it).

Regards,

Bob Peterson
Red Hat Cluster Suite