[Linux-cluster] Set time on cluster

[Dryden, Tom]

Thank you, we will look into upgrading.



-----Original Message-----
From: Jan Friesse [mailto:jfriesse at redhat.com] 
Sent: Wednesday, October 24, 2012 3:24 AM
To: linux clustering
Cc: Dryden, Tom
Subject: Re: [Linux-cluster] Set time on cluster

We have support for monotonic timer since 0.80.6-33 (so RHEL >= 5.8).
This should solve problems you are describing. What version are you
using? If it's older then 0.80.6-33, try upgrade.

Regards,
  Honza

Dryden, Tom napsal(a):
>  
> 
> Hello All
> 
> I had a problem when time adjusted that caused node evictions.   The
> sequence of events are in the log clip below. 
> 
>  The events were,
> 
> 1. ntpd restarted
> 
> 2. ntpd adjusted time  +155 sec
> 
> 3. openais thinks it has lost communication and begins to evict nodes.
> 
>  
> 
> My question is how do I adjust the time without causing the cluster to

> get confused?
> 
>  
> 
> Thanks for your assistance. 
> 
> Tom
> 
>  
> 
>  
> 
> LOG CLIP:
> 
> Oct 22 16:05:02 bpas-j5 ntpd[27399]: Listening on interface bond0:9,
> 172.26.5.21#123 Enabled
> 
> Oct 22 16:05:02 bpas-j5 ntpd[27399]: kernel time sync status 0040
> 
> Oct 22 16:05:03 bpas-j5 ntpd[27399]: frequency initialized 40.201 PPM 
> from /var/lib/ntp/drift
> 
> Oct 22 16:08:15 bpas-j5 ntpd[27399]: synchronized to 169.229.70.201, 
> stratum 3
> 
> Oct 22 16:10:50 bpas-j5 ntpd[27399]: time reset +155.075282 s
> 
> Oct 22 16:10:50 bpas-j5 ntpd[27399]: kernel time sync enabled 0001
> 
> Oct 22 16:10:50 bpas-j5 openais[7238]: [TOTEM] The token was lost in 
> the OPERATIONAL state.
> 
> Oct 22 16:10:50 bpas-j5 openais[7238]: [TOTEM] Receive multicast 
> socket recv buffer size (288000 bytes).
> 
> Oct 22 16:10:50 bpas-j5 openais[7238]: [TOTEM] Transmit multicast 
> socket send buffer size (262142 bytes).
> 
> Oct 22 16:10:50 bpas-j5 openais[7238]: [TOTEM] entering GATHER state 
> from 2.
> 
> Oct 22 16:10:55 bpas-j5 openais[7238]: [TOTEM] entering GATHER state 
> from 0.
> 
> Oct 22 16:10:55 bpas-j5 openais[7238]: [TOTEM] Creating commit token 
> because I am the rep.
> 
> Oct 22 16:10:55 bpas-j5 openais[7238]: [TOTEM] Saving state aru 33 
> high seq received 33
> 
> Oct 22 16:10:55 bpas-j5 openais[7238]: [TOTEM] Storing new sequence id

> for ring 1f48
> 
> Oct 22 16:10:55 bpas-j5 openais[7238]: [TOTEM] entering COMMIT state.
> 
> Oct 22 16:10:55 bpas-j5 openais[7238]: [TOTEM] entering RECOVERY
state.
> 
> Oct 22 16:10:55 bpas-j5 openais[7238]: [TOTEM] position [0] member
> 172.26.5.11:
> 
> Oct 22 16:10:55 bpas-j5 openais[7238]: [TOTEM] previous ring seq 8004 
> rep 172.26.5.11
> 
> Oct 22 16:10:55 bpas-j5 openais[7238]: [TOTEM] aru 33 high delivered 
> 33 received flag 1
> 
> Oct 22 16:10:55 bpas-j5 openais[7238]: [TOTEM] Did not need to 
> originate any messages in recovery.
> 
> Oct 22 16:10:55 bpas-j5 openais[7238]: [TOTEM] Sending initial ORF 
> token
> 
> Oct 22 16:10:55 bpas-j5 openais[7238]: [CLM  ] CLM CONFIGURATION 
> CHANGE
> 
> Oct 22 16:10:55 bpas-j5 openais[7238]: [CLM  ] New Configuration:
> 
> Oct 22 16:10:55 bpas-j5 kernel: dlm: closing connection to node 2
> 
> Oct 22 16:10:55 bpas-j5 openais[7238]: [CLM  ]  r(0) ip(172.26.5.11)
> 
> Oct 22 16:10:55 bpas-j5 openais[7238]: [CLM  ] Members Left:
> 
> Oct 22 16:10:55 bpas-j5 openais[7238]: [CLM  ]  r(0) ip(172.26.6.11)
> 
>  
> 
> 
> 
>