[Linux-cluster] openais issue
Paras pradhan
pradhanparas at gmail.com
Tue Sep 29 15:45:37 UTC 2009
Withe log_level of 3 I got only this
Sep 29 10:31:31 cvtst1 rgmanager: [7170]: <notice> Shutting down
Cluster Service Manager...
Sep 29 10:31:31 cvtst1 clurgmgrd[6673]: <notice> Shutting down
Sep 29 10:31:41 cvtst1 clurgmgrd[6673]: <notice> Shutdown complete, exiting
Sep 29 10:31:41 cvtst1 rgmanager: [7170]: <notice> Cluster Service
Manager is stopped.
Sep 29 10:31:42 cvtst1 clurgmgrd[7224]: <notice> Resource Group
Manager Starting
Sep 29 10:39:06 cvtst1 rgmanager: [10327]: <notice> Shutting down
Cluster Service Manager...
Sep 29 10:39:16 cvtst1 rgmanager: [10327]: <notice> Cluster Service
Manager is stopped.
Sep 29 10:39:16 cvtst1 clurgmgrd[10380]: <notice> Resource Group
Manager Starting
Sep 29 10:39:52 cvtst1 clurgmgrd[10380]: <notice> Member 1 shutting down
I do not know what the last line means.
rgmanager version I am running is:
rgmanager-2.0.52-1.el5.centos
I don't what has gone wrong.
Thanks
Paras.
On Mon, Sep 28, 2009 at 6:41 PM, brem belguebli
<brem.belguebli at gmail.com> wrote:
> you mean it stopped successfully on all the nodes but it is failing to
> start only on node cvtst1 ?
>
> look at the following page to make rgmanager more verbose. It 'll
> help debug....
>
> http://sources.redhat.com/cluster/wiki/RGManager
>
> at Logging Configuration section
>
>
>
>
> 2009/9/29 Paras pradhan <pradhanparas at gmail.com>:
>> Brem,
>>
>> When I try to restart rgmanager on all the nodes, this time i do not
>> see rgmanager running on the first node. But I do see on other 2
>> nodes.
>>
>> Log on the first node:
>>
>> Sep 28 18:13:58 cvtst1 clurgmgrd[24099]: <notice> Resource Group
>> Manager Starting
>> Sep 28 18:17:29 cvtst1 rgmanager: [24627]: <notice> Shutting down
>> Cluster Service Manager...
>> Sep 28 18:17:29 cvtst1 clurgmgrd[24099]: <notice> Shutting down
>> Sep 28 18:17:39 cvtst1 clurgmgrd[24099]: <notice> Shutdown complete, exiting
>> Sep 28 18:17:39 cvtst1 rgmanager: [24627]: <notice> Cluster Service
>> Manager is stopped.
>> Sep 28 18:17:40 cvtst1 clurgmgrd[24679]: <notice> Resource Group
>> Manager Starting
>>
>> -
>> It seems service is running , but I do not see rgmanger running using clustat
>>
>>
>> Don't know what is going on.
>>
>> Thanks
>> Paras.
>>
>>
>> On Mon, Sep 28, 2009 at 5:46 PM, brem belguebli
>> <brem.belguebli at gmail.com> wrote:
>>> Paras,
>>>
>>> Another thing, it would have been more interesting to have a start
>>> DEBUG not a stop.
>>>
>>> That's why I was asking you to first stop the vm manually on all your
>>> nodes, stop eventually rgmanager on all the nodes to reset the
>>> potential wrong states you may have, restart rgmanager.
>>>
>>> If your VM is configured to autostart, this will make it start.
>>>
>>> It should normally fail (as it does now). Send out your newly created
>>> DEBUG file.
>>>
>>> 2009/9/29 brem belguebli <brem.belguebli at gmail.com>:
>>>> Hi Paras,
>>>>
>>>>
>>>> I don't know the xen/cluster combination well, but if I do remember
>>>> well, I think I've read somewhere that when using xen you have to
>>>> declare the use_virsh=0 key in the VM definition in the cluster.conf.
>>>>
>>>> This would make rgmanager use xm commands instead of virsh
>>>> The DEBUG output shows clearly that you are using virsh to manage your
>>>> VM instead of xm commands.
>>>> Check out the RH docs about virtualization
>>>>
>>>> I'm not a 100% sure about that, I may be completely wrong.
>>>>
>>>> Brem
>>>>
>>>> 2009/9/28 Paras pradhan <pradhanparas at gmail.com>:
>>>>> The only thing I noticed is the message after stopping the vm using xm
>>>>> in all nodes and starting using clusvcadm is
>>>>>
>>>>> "Virtual machine guest1 is blocked"
>>>>>
>>>>> The whole DEBUG file is attached.
>>>>>
>>>>>
>>>>> Thanks
>>>>> Paras.
>>>>>
>>>>> On Fri, Sep 25, 2009 at 5:53 PM, brem belguebli
>>>>> <brem.belguebli at gmail.com> wrote:
>>>>>> There's a problem with the script that is called by rgmanager to start
>>>>>> the VM, I don't know what causes it
>>>>>>
>>>>>> May be you should try something like :
>>>>>>
>>>>>> 1) stop the VM on all nodes with xm commands
>>>>>> 2) edit the /usr/share/cluster/vm.sh script and add the following
>>>>>> lines (after the #!/bin/bash ):
>>>>>> exec >/tmp/DEBUG 2>&1
>>>>>> set -x
>>>>>> 3) start the VM with clusvcadm -e vm:guest1
>>>>>>
>>>>>> It should fail as it did before.
>>>>>>
>>>>>> edit the the /tmp/DEBUG file and you will be able to see where it
>>>>>> fails (it may generate a lot of debug)
>>>>>>
>>>>>> 4) remove the debug lines from /usr/share/cluster/vm.sh
>>>>>>
>>>>>> Post the DEBUG file if you're not able to see where it fails.
>>>>>>
>>>>>> Brem
>>>>>>
>>>>>> 2009/9/26 Paras pradhan <pradhanparas at gmail.com>:
>>>>>>> No I am not manually starting not using automatic init scripts.
>>>>>>>
>>>>>>> I started the vm using: clusvcadm -e vm:guest1
>>>>>>>
>>>>>>> I have just stopped using clusvcadm -s vm:guest1. For few seconds it
>>>>>>> says guest1 started . But after a while I can see the guest1 on all
>>>>>>> three nodes.
>>>>>>>
>>>>>>> clustat says:
>>>>>>>
>>>>>>> Service Name Owner (Last)
>>>>>>> State
>>>>>>> ------- ---- ----- ------
>>>>>>> -----
>>>>>>> vm:guest1 (none)
>>>>>>> stopped
>>>>>>>
>>>>>>> But I can see the vm from xm li.
>>>>>>>
>>>>>>> This is what I can see from the log:
>>>>>>>
>>>>>>>
>>>>>>> Sep 25 17:19:01 cvtst1 clurgmgrd[4298]: <notice> start on vm "guest1"
>>>>>>> returned 1 (generic error)
>>>>>>> Sep 25 17:19:01 cvtst1 clurgmgrd[4298]: <warning> #68: Failed to start
>>>>>>> vm:guest1; return value: 1
>>>>>>> Sep 25 17:19:01 cvtst1 clurgmgrd[4298]: <notice> Stopping service vm:guest1
>>>>>>> Sep 25 17:19:02 cvtst1 clurgmgrd[4298]: <notice> Service vm:guest1 is
>>>>>>> recovering
>>>>>>> Sep 25 17:19:15 cvtst1 clurgmgrd[4298]: <notice> Recovering failed
>>>>>>> service vm:guest1
>>>>>>> Sep 25 17:19:16 cvtst1 clurgmgrd[4298]: <notice> start on vm "guest1"
>>>>>>> returned 1 (generic error)
>>>>>>> Sep 25 17:19:16 cvtst1 clurgmgrd[4298]: <warning> #68: Failed to start
>>>>>>> vm:guest1; return value: 1
>>>>>>> Sep 25 17:19:16 cvtst1 clurgmgrd[4298]: <notice> Stopping service vm:guest1
>>>>>>> Sep 25 17:19:17 cvtst1 clurgmgrd[4298]: <notice> Service vm:guest1 is
>>>>>>> recovering
>>>>>>>
>>>>>>>
>>>>>>> Paras.
>>>>>>>
>>>>>>> On Fri, Sep 25, 2009 at 5:07 PM, brem belguebli
>>>>>>> <brem.belguebli at gmail.com> wrote:
>>>>>>>> Have you started your VM via rgmanager (clusvcadm -e vm:guest1) or
>>>>>>>> using xm commands out of cluster control (or maybe a thru an
>>>>>>>> automatic init script ?)
>>>>>>>>
>>>>>>>> When clustered, you should never be starting services (manually or
>>>>>>>> thru automatic init script) out of cluster control
>>>>>>>>
>>>>>>>> The thing would be to stop your vm on all the nodes with the adequate
>>>>>>>> xm command (not using xen myself) and try to start it with clusvcadm.
>>>>>>>>
>>>>>>>> Then see if it is started on all nodes (send clustat output)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 2009/9/25 Paras pradhan <pradhanparas at gmail.com>:
>>>>>>>>> Ok. Please see below. my vm is running on all nodes though clustat
>>>>>>>>> says it is stopped.
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> [root at cvtst1 ~]# clustat
>>>>>>>>> Cluster Status for test @ Fri Sep 25 16:52:34 2009
>>>>>>>>> Member Status: Quorate
>>>>>>>>>
>>>>>>>>> Member Name ID Status
>>>>>>>>> ------ ---- ---- ------
>>>>>>>>> cvtst2 1 Online, rgmanager
>>>>>>>>> cvtst1 2 Online,
>>>>>>>>> Local, rgmanager
>>>>>>>>> cvtst3 3 Online, rgmanager
>>>>>>>>>
>>>>>>>>> Service Name Owner (Last)
>>>>>>>>> State
>>>>>>>>> ------- ---- ----- ------
>>>>>>>>> -----
>>>>>>>>> vm:guest1 (none)
>>>>>>>>> stopped
>>>>>>>>> [root at cvtst1 ~]#
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---
>>>>>>>>> o/p of xm li on cvtst1
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> [root at cvtst1 ~]# xm li
>>>>>>>>> Name ID Mem(MiB) VCPUs State Time(s)
>>>>>>>>> Domain-0 0 3470 2 r----- 28939.4
>>>>>>>>> guest1 7 511 1 -b---- 7727.8
>>>>>>>>>
>>>>>>>>> o/p of xm li on cvtst2
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> [root at cvtst2 ~]# xm li
>>>>>>>>> Name ID Mem(MiB) VCPUs State Time(s)
>>>>>>>>> Domain-0 0 3470 2 r----- 31558.9
>>>>>>>>> guest1 21 511 1 -b---- 7558.2
>>>>>>>>> ---
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> Paras.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Sep 25, 2009 at 4:22 PM, brem belguebli
>>>>>>>>> <brem.belguebli at gmail.com> wrote:
>>>>>>>>>> It looks like no.
>>>>>>>>>>
>>>>>>>>>> can you send an output of clustat of when the VM is running on
>>>>>>>>>> multiple nodes at the same time?
>>>>>>>>>>
>>>>>>>>>> And by the way, another one after having stopped (clusvcadm -s vm:guest1) ?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2009/9/25 Paras pradhan <pradhanparas at gmail.com>:
>>>>>>>>>>> Anyone having issue as mine? Virtual machine service is not being
>>>>>>>>>>> properly handled by the cluster.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>> Paras.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Sep 21, 2009 at 9:55 AM, Paras pradhan <pradhanparas at gmail.com> wrote:
>>>>>>>>>>>> Ok.. here is my cluster.conf file
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> [root at cvtst1 cluster]# more cluster.conf
>>>>>>>>>>>> <?xml version="1.0"?>
>>>>>>>>>>>> <cluster alias="test" config_version="9" name="test">
>>>>>>>>>>>> <fence_daemon clean_start="0" post_fail_delay="0" post_join_delay="3"/>
>>>>>>>>>>>> <clusternodes>
>>>>>>>>>>>> <clusternode name="cvtst2" nodeid="1" votes="1">
>>>>>>>>>>>> <fence/>
>>>>>>>>>>>> </clusternode>
>>>>>>>>>>>> <clusternode name="cvtst1" nodeid="2" votes="1">
>>>>>>>>>>>> <fence/>
>>>>>>>>>>>> </clusternode>
>>>>>>>>>>>> <clusternode name="cvtst3" nodeid="3" votes="1">
>>>>>>>>>>>> <fence/>
>>>>>>>>>>>> </clusternode>
>>>>>>>>>>>> </clusternodes>
>>>>>>>>>>>> <cman/>
>>>>>>>>>>>> <fencedevices/>
>>>>>>>>>>>> <rm>
>>>>>>>>>>>> <failoverdomains>
>>>>>>>>>>>> <failoverdomain name="myfd1" nofailback="0" ordered="1" restricted="0">
>>>>>>>>>>>> <failoverdomainnode name="cvtst2" priority="3"/>
>>>>>>>>>>>> <failoverdomainnode name="cvtst1" priority="1"/>
>>>>>>>>>>>> <failoverdomainnode name="cvtst3" priority="2"/>
>>>>>>>>>>>> </failoverdomain>
>>>>>>>>>>>> </failoverdomains>
>>>>>>>>>>>> <resources/>
>>>>>>>>>>>> <vm autostart="1" domain="myfd1" exclusive="0" max_restarts="0"
>>>>>>>>>>>> name="guest1" path="/vms" recovery="r
>>>>>>>>>>>> estart" restart_expire_time="0"/>
>>>>>>>>>>>> </rm>
>>>>>>>>>>>> </cluster>
>>>>>>>>>>>> [root at cvtst1 cluster]#
>>>>>>>>>>>> ------
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks!
>>>>>>>>>>>> Paras.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Sep 20, 2009 at 9:44 AM, Volker Dormeyer <volker at ixolution.de> wrote:
>>>>>>>>>>>>> On Fri, Sep 18, 2009 at 05:08:57PM -0500,
>>>>>>>>>>>>> Paras pradhan <pradhanparas at gmail.com> wrote:
>>>>>>>>>>>>>> I am using cluster suite for HA of xen virtual machines. Now I am
>>>>>>>>>>>>>> having another problem. When I start the my xen vm in one node, it
>>>>>>>>>>>>>> also starts on other nodes. Which daemon controls this?
>>>>>>>>>>>>>
>>>>>>>>>>>>> This is usually done bei clurgmgrd (which is part of the rgmanager
>>>>>>>>>>>>> package). To me, this sounds like a configuration problem. Maybe,
>>>>>>>>>>>>> you can post your cluster.conf?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Volker
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Linux-cluster mailing list
>>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Linux-cluster mailing list
>>>>>>>>> Linux-cluster at redhat.com
>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Linux-cluster mailing list
>>>>>>>> Linux-cluster at redhat.com
>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Linux-cluster mailing list
>>>>>>> Linux-cluster at redhat.com
>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Linux-cluster mailing list
>>>>>> Linux-cluster at redhat.com
>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>>
>>>>>
>>>>> --
>>>>> Linux-cluster mailing list
>>>>> Linux-cluster at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>>>
>>>>
>>>
>>> --
>>> Linux-cluster mailing list
>>> Linux-cluster at redhat.com
>>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>>
>>
>> --
>> Linux-cluster mailing list
>> Linux-cluster at redhat.com
>> https://www.redhat.com/mailman/listinfo/linux-cluster
>>
>
> --
> Linux-cluster mailing list
> Linux-cluster at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster
>
More information about the Linux-cluster
mailing list