<html> <head> <meta content="text/html; charset=windows-1252" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> <div class="moz-cite-prefix">On 5/19/15 12:17 AM, Ludwig Krispenz wrote: </div> <blockquote cite="mid:555AE399.1050901@redhat.com" type="cite"> <meta content="text/html; charset=windows-1252" http-equiv="Content-Type"> <div class="moz-cite-prefix">On 05/19/2015 08:58 AM, thierry bordaz wrote: </div> <blockquote cite="mid:555ADEFF.5090304@redhat.com" type="cite"> <meta content="text/html; charset=windows-1252" http-equiv="Content-Type"> <div class="moz-cite-prefix">On 05/19/2015 07:47 AM, Martin Kosek wrote: </div> <blockquote cite="mid:555ACE84.3030105@redhat.com" type="cite">On 05/19/2015 03:23 AM, Janelle wrote: <blockquote type="cite">Once again, replication/sync has been lost. I really wish the product was more stable, it is so much potential and yet. Servers running for 6 days no issues. No new accounts or changes (maybe a few users changing passwords) and again, 5 out of 16 servers are no longer in sync. I can test it easily by adding an account and then waiting a few minutes, then run "ipa user-show --all username" on all the servers, and only a few of them have the account. I have now waited 15 minutes, still no luck. Oh well.. I guess I will go look at alternatives. I had such high hopes for this tool. Thanks so much everyone for all your help in trying to get things stable, but for whatever reason, there is a random loss of sync among the servers and obviously this is not acceptable. </blockquote> Hello Janelle, I am very sorry to hear about your troubles. Would you be still OK with helping us (mostly Ludwig and Thierry) investigate what is the root cause of the instability of the replication agreements? This is obviously something that should not be happening at this rate as in your deployment, so I would really like to be able to identity and fix this issue in the 389 DS. </blockquote> Hello Janelle, I can only join my voice to Martin to say how I am sorry to read this. Would you turn on replication logging level (8192) on the master/consumer and provide us the logs(access/error) and config (dse.ldif). The master is the instance where you can see the update and the that is linked (replica agreement) to a replica(aka consumer) where the update is not received. </blockquote> what puzzles me most, is that replication is working for quite some time and then breaks, so we need to find out about the dynamics which lead to that state. You reported errors about invalid credentials or about a bind dn entry not found, these credentials don't get changed by ds or entries are not deleted by ds, so what triggers these changes. also for the suggestion by Thierry to debug, we need to determine where replication breaks, if you add an account and it is propageted to some servers and not to others, where does it stop ? This depends on your replication topology, you said in anotehr post that you have a ring topology, does it mean all 16 servers are conencted in a ring only, and if two links break the topology is disconnected ? <blockquote cite="mid:555ADEFF.5090304@redhat.com" type="cite"> thanks thierry </blockquote> </blockquote> Let me see about getting some debug logs going to provide more info. As for topology -- yes, ring, but also within the DC - the 3 servers are connected in an internal ring. There have been no outages on the WAN connections, as I have logs showing network data at all times, so this is not an issue. If I did lose a WAN, dozens of other inter-DC apps would blow up too, and they have not. However, I guess you are right, I have not provided enough logging data to help diagnose this. Let me see what I can do. Not sure if this helps -- I do try and do all updates from a single master, never from different ones. Users are also forced to the same master to change passwords and update things. So the "source" of changes is always the same. Time to go do some log enabling... ~J </body> </html>