[Linux-cluster] Fencing problem w/ 2-node VM when a VM host dies

[Kelvin Edmison]

On 12/03/2015 06:14 PM, Digimer wrote:
> On 03/12/15 02:19 PM, Kelvin Edmison wrote:
>> I am hoping that someone can help me understand the problems I'm having
>> with linux clustering for VMs.
>>
>> I am clustering 2 VMs on two separate VM hosts, trying to ensure that a
>> service is always available.  The hosts and guests are both RHEL 6.7.
>> The goal is to have only one of the two VMs running at a time.
>>
>> The configuration works when we test/simulate VM deaths and graceful VM
>> host shutdowns, and administrative switchovers (i.e. clusvcadm -r ).
>>
>> However, when we simulate the sudden isolation of host A (e.g. ifdown
>> eth0), two things happen
>> 1) the VM on host B does not start, and repeated fence_xvm errors appear
>> in the logs on host B
>> 2) when the 'failed' node is returned to service, the cman service on
>> host B dies.
> If the node's host is dead, then there is no way for the survivor to
> determine the state of the lost VM node. The cluster is not allowed to
> take "no answer" as confirmation of fence success.
>
> If your hosts have IPMI, then you could add fence_ipmilan as a backup
> method where, if fence_xvm fails, it moves on and reboots the host itself.

Thank you for the suggestion.  The hosts do have ipmi.  I'll explore it 
but I'm a little concerned about what it means for the other 
non-clustered VM workloads that exist on these two servers.

Do you have any thoughts as to why host B's cman process is dying when 
'host A' returns?

Thanks,
   Kelvin