[Linux-cluster] ricci is very unstable in one nodes

[fosiul alam]

Hi
I have 4 nodes cluster,
It was running fine. but today one nodes is giving trouble

>From luci Gui interface, when i try to relocate service into this node and
trying to relocate from this nodes to another nodes

from luci gui interface, its showing :

Unable to retrieve batch 1908047789 status from beaver.domain.local:11111:
clusvcadm start failed to start httpd1: Starting cluster service "httpd1" on
node "http1.domain.local" -- You will be redirected in 5 seconds.also

*The ricci agent for this node is unresponsive. Node-specific information is
not available at this time.  :

but ricci is running on problematic node ,
ricci     7324  0.0  0.1  58876  2932 ?        S<s  14:40   0:00 ricci -u
101

 there is not any firewall running.

 iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain RH-Firewall-1-INPUT (0 references)
target     prot opt source               destination

port 11111 is runningg

netstat -an | grep 11111
tcp        0      0 0.0.0.0:11111               0.0.0.0:*
LISTEN


but still ricci is very unstable , and i cant relocate any service on this
node or i cant relocate any service away from this node.

from problematic node if i type this

 clustat
Cluster Status for ng1 @ Thu Sep 23 20:24:02 2010
Member Status: Quorate

 Member Name                             ID   Status
 ------ ----                             ---- ------
 beaver.xxx.local                  1 Online, rgmanager         ::: luci is
running from this server
 publicdns1.xxxx.local              2 Online, rgmanager
 http1.xxxx.local                   3 Online, Local, rgmanager
 mail01.xxxxx.local                  4 Online, rgmanager

 Service Name                   Owner (Last)                   State
 ------- ----                   ----- ------                   -----
 service:httpd1                 mail01.xxxx.local     started
 service:mysql-server           http1.xxxx.local      started
------------------- this is the problematic node
 service:public-dns             publicdns1.xxxxxx.local started

I cant move that service mysql-server from this node or cant relocate any
service on this node ..
I am very confused.

what shall i do  to fix this issue ??

thanks for your advise.



*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/linux-cluster/attachments/20100923/f4009045/attachment.htm>