<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Sep 26, 2016, at 4:07 PM, Timothy Geier <<a href="mailto:tgeier@accertify.com" class="">tgeier@accertify.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br class="Apple-interchange-newline">
On Sep 26, 2016, at 2:17 PM, Timothy Geier <<a href="mailto:tgeier@accertify.com" class="">tgeier@accertify.com</a>> wrote:<br class="">
<br class="">
This issue started when trying to remove a user; ipa user-del showed “operation failed” and the user was not removed. The same ipa user-del command was performed on a replica and completed successfully, but it was then immediately apparent that this change
did not replicate anywhere else. All of the replicas then were re-initalized using "ipa-replica-manage re-initialize” and now the LDAP trees/users are consistent though no further changes have been made.<br class="">
<br class="">
The slapd error logs are showing repeated instances of<br class="">
<br class="">
DSRetroclPlugin - replog: an error occured while adding change number 112697, dn = changenumber=112697,cn=changelog: Already exists.<br class="">
retrocl-plugin - retrocl_postob: operation failure [68]<br class="">
<br class="">
Package versions are<br class="">
ipa-server-4.2.0-15.0.1.el7.centos.6.1.x86_64<br class="">
and<span class="Apple-converted-space"> </span><br class="">
389-ds-base-1.3.4.0-29.el7_2.x86_64<br class="">
<br class="">
ipa-replica-manage list-ruv<br class="">
ipa: WARNING: session memcached servers not running<br class="">
unable to decode: {replica 11} 56044ef50000000b0000 56044ef50000000b0000<br class="">
unable to decode: {replica 7} 561f17ba000800070000 561f17ba000800070000<br class="">
unable to decode: {replica 5} 561f17bc000300050000 561f17bc000300050000<br class="">
unable to decode: {replica 9} 561f17ba000a00090000 561f17ba000a00090000<br class="">
unable to decode: {replica 4} 561f17ba000300040000 561f17ba000300040000<span class="Apple-converted-space"> </span><br class="">
(These are likely leftovers from the previous incarnation of these servers on a RHEL6-like setup)<br class="">
ipa07:389: 16<br class="">
ipa02:389: 13<br class="">
ipa03:389: 14<br class="">
ipa01:389: 12<br class="">
ipa04:389: 15<br class="">
ipa05:389: 17<br class="">
<br class="">
Thanks much,<br class="">
</blockquote>
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">After
not taking any action, this error has stopped but has been replaced with</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">[26/Sep/2016:15:54:54
-0500] NSMMReplicationPlugin - agmt="cn=meToipa03" (ipa03:389): Missing data encountered</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">[26/Sep/2016:15:54:54
-0500] NSMMReplicationPlugin - agmt="cn=meToipa03" (ipa03:389): Incremental update failed and requires administrator action</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">for
all of the replicas and things are slightly out of sync everywhere. </span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Is
the best course of action here to declare one a new master and do a ipa-replica-manage re-initialize to all of the others from that one?</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<br class="">
</div>
</blockquote>
<br class="">
</div>
<div>After doing some testing, that’s exactly what we did and replication is now working again. It is odd that the DSRetroclPlugin errors stopped on their own (after approximately 3 hours); the only action taken there was looking at the cn=changelog base using
ldapvi to see what number it was on but that has to be a sheer coincidence; absolutely no changes were made. </div>
<div><br class="">
</div>
<div>We’re also still unsure what caused this; our best theory at the moment is a race condition where everything that could have gone wrong at that exact moment did..is there any validity to this?</div>
<div><br class="">
</div>
<div>Thanks,</div>
</body>
</html>
<pre>
"This message and any attachments may contain confidential information. If you
have received this message in error, any use or distribution is prohibited.
Please notify us by reply e-mail if you have mistakenly received this message,
and immediately and permanently delete it and any attachments. Thank you."</pre>