<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div> On May 29, 2015, at 00:41, thierry bordaz <<a href="mailto:tbordaz@redhat.com">tbordaz@redhat.com</a>> wrote: </div><blockquote type="cite"><div> <meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type"> <div class="moz-cite-prefix">On 05/29/2015 08:16 AM, Christoph Kaminski wrote: </div> <blockquote cite="mid:OF2BC78040.A4664FCF-ONC1257E54.0020B1D9-C1257E54.00227497@biotronik.com" type="cite"><tt><a class="moz-txt-link-abbreviated" href="mailto:freeipa-users-bounces@redhat.com">freeipa-users-bounces@redhat.com</a> schrieb am 28.05.2015 13:23:26: > Von: Alexander Frolushkin <a class="moz-txt-link-rfc2396E" href="mailto:Alexander.Frolushkin@megafon.ru"><Alexander.Frolushkin@megafon.ru></a></tt> <tt>> An: "'thierry bordaz'" <a class="moz-txt-link-rfc2396E" href="mailto:tbordaz@redhat.com"><tbordaz@redhat.com></a></tt> <tt>> Kopie: <a class="moz-txt-link-rfc2396E" href="mailto:freeipa-users@redhat.com">"freeipa-users@redhat.com"</a> <a class="moz-txt-link-rfc2396E" href="mailto:freeipa-users@redhat.com"><freeipa-users@redhat.com></a></tt> <tt>> Datum: 28.05.2015 13:24</tt> <tt>> Betreff: Re: [Freeipa-users] Haunted servers?</tt> <tt>> Gesendet von: <a class="moz-txt-link-abbreviated" href="mailto:freeipa-users-bounces@redhat.com">freeipa-users-bounces@redhat.com</a></tt> <tt>> > Unfortunately, after a couple of minutes, on two of three servers > error comes back in little changed form: > # ipa-replica-manage list-ruv > unable to decode: {replica 16} > .... > > Before cleanruv it looked like: > # ipa-replica-manage list-ruv > unable to decode: {replica 16} 548a8126000000100000 548a8126000000100000 > .... > > And one server seems to be fixed completely. > > WBR, > Alexander Frolushkin > > </tt> <tt>we had the same problem (and some more) and yesterday we have successfully cleaned the gohst rid's</tt> <tt>our fix:</tt> </blockquote> Hi Christoph, THanks for sharing this procedure. This bug is difficult to workaround and that is a good idea to write it down. <blockquote cite="mid:OF2BC78040.A4664FCF-ONC1257E54.0020B1D9-C1257E54.00227497@biotronik.com" type="cite"> <tt>1. stop all cleanallruv Tasks, if it works with ipa-replica-manage abort-clean-ruv. It hasnt worked here. We have done it manually on ALL replicas with:</tt> <tt> a) replica stop</tt> <tt> b) delete all nsds5ReplicaClean from /etc/dirsrv/slapd-HSO/dse.ldif</tt> <tt> c) replica start</tt> </blockquote> Yes the ability to abort clean ruv hits the same retry issue that cleanallruv. It has been addressed with <a class="moz-txt-link-freetext" href="https://fedorahosted.org/389/ticket/48154">https://fedorahosted.org/389/ticket/48154</a> <blockquote cite="mid:OF2BC78040.A4664FCF-ONC1257E54.0020B1D9-C1257E54.00227497@biotronik.com" type="cite"><tt>2. prepare on EACH ipa a cleanruv ldif file with ALL ghost rids inside (really ALL from all ipa replicas, we has had some rids only on some replicas...)</tt> <tt>Example:</tt> <tt>dn: cn=replica,cn=dc\3Dexample,cn=mapping tree,cn=config</tt> <tt>changetype: modify</tt> <tt>replace: nsds5task</tt> <tt>nsds5task:CLEANRUV11</tt> <tt>dn: cn=replica,cn=dc\3Dexample,cn=mapping tree,cn=config</tt> <tt>changetype: modify</tt> <tt>replace: nsds5task</tt> <tt>nsds5task:CLEANRUV22</tt> <tt>dn: cn=replica,cn=dc\3Dexample,cn=mapping tree,cn=config</tt> <tt>changetype: modify</tt> <tt>replace: nsds5task</tt> <tt>nsds5task:CLEANRUV37</tt> <tt>...</tt> </blockquote> It should work but I would prefer to do it in an other order. We need to clean a specific RID, on all replica, at the same time. We do not need to clean all RIDs at the same time. Having several CLEANRUV in parallel for differents RID should work but I do not know how much it has been tested that way. So I would recommend to clean, in parallel on all replicas, RID 11. Then when it is completed, RID 22. Then RID 37. <blockquote cite="mid:OF2BC78040.A4664FCF-ONC1257E54.0020B1D9-C1257E54.00227497@biotronik.com" type="cite"> <tt>3. do a "ldapmodify -h 127.0.0.1 -D "cn=Directory Manager" -W -x -f $your-cleanruv-file.ldif" on all replicas AT THE SAME TIME :) we used terminator for it (</tt><a moz-do-not-send="true" href="https://launchpad.net/terminator"><tt>https://launchpad.net/terminator</tt></a><tt>). You can open multiple shell windows inside one window and send to all at the same time the same commands...</tt> </blockquote> same remark I would split your-cleanruv-file.ldif into three files cleanruv-11-file.ldif,... <blockquote cite="mid:OF2BC78040.A4664FCF-ONC1257E54.0020B1D9-C1257E54.00227497@biotronik.com" type="cite"> <tt>4. we have done a re-initialize of each IPA from our first master</tt> </blockquote> Do you mean a total init ? I do not see a real need for that. If you are ready to reinit all replicas, there is a very simple way to get rid of all these ghost RIDs. <ul> <li>Select the "good" master that is having all the updates </li> <li>do a ldif export without the replication data</li> <li>do a ldif import of exported file</li> <li>do online reinit of the full topology, cascading from the "good" master down to the "consumers"</li> </ul> Most of the time we try to avoid asking a full reinit of the topology because DB are large. <blockquote cite="mid:OF2BC78040.A4664FCF-ONC1257E54.0020B1D9-C1257E54.00227497@biotronik.com" type="cite"> <tt>5. restart of all replicas</tt> <tt>we are not sure about the point 3 and 4. Maybe they are not necessary, but we have done it.</tt> <tt>If something fails look at defect LDAP entries in whole ldap, we have had some entries with 'nsunique-$HASH' after the 'normal' name. We have deleted them.</tt> </blockquote> do you mean entries with 'nsuniqueid' attribute in the RDN. This could be create during replication conflicts when updates are received in parallele on different replicas. thanks thierry <blockquote cite="mid:OF2BC78040.A4664FCF-ONC1257E54.0020B1D9-C1257E54.00227497@biotronik.com" type="cite"><tt> </tt>MfG Christoph Kaminski </blockquote> </div></blockquote><blockquote type="cite"><div>-- Manage your subscription for the Freeipa-users mailing list: <a href="https://www.redhat.com/mailman/listinfo/freeipa-users">https://www.redhat.com/mailman/listinfo/freeipa-users</a> Go to <a href="http://freeipa.org">http://freeipa.org</a> for more info on the project</div></blockquote> <div>Looks like I'll be giving this a try. So glad someone else is seeing exactly the same issues. Hopefully soon we can find the cause.</div><div> </div><div>~J</div></body></html>