[Freeipa-users] IPA Replication Status

Choudhury, Suhail Suhail.Choudhury at bskyb.com
Wed Jul 23 11:58:24 UTC 2014


I have the following errors on different boxes:

[root at recsds1 sch32]# tail -f /var/log/dirsrv/slapd-RECS-BSKYB-COM/errors
[23/Jul/2014:12:28:54 +0100] NSMMReplicationPlugin - CleanAllRUV Task: Replicas have not been cleaned yet, retrying in 10 seconds
[23/Jul/2014:12:29:06 +0100] NSMMReplicationPlugin - CleanAllRUV Task: Waiting for all the replicas to finish cleaning...
[23/Jul/2014:12:29:06 +0100] NSMMReplicationPlugin - CleanAllRUV Task: Not all replicas finished cleaning, retrying in 10 seconds
[23/Jul/2014:12:29:16 +0100] NSMMReplicationPlugin - CleanAllRUV Task: Not all replicas finished cleaning, retrying in 20 seconds
[23/Jul/2014:12:29:36 +0100] NSMMReplicationPlugin - CleanAllRUV Task: Not all replicas finished cleaning, retrying in 40 seconds

[root at recsds3 sch32]# tail -f /var/log/dirsrv/slapd-RECS-BSKYB-COM/errors
[23/Jul/2014:12:52:10 +0100] agmt="cn=meTorecsds2.bskyb.com" (recsds2:389) - Can't locate CSN 53c7ba27000000100000 in the changelog (DB rc=-30988). The consumer may need to be reinitialized.
[23/Jul/2014:12:52:10 +0100] NSMMReplicationPlugin - agmt="cn=meTorecsds2.bskyb.com" (recsds2:389): changelog iteration code returned a dummy entry with csn 53c7c6b1000200100000, skipping ...
[23/Jul/2014:12:52:13 +0100] agmt="cn=meTorecsds4.bskyb.com" (recsds4:389) - Can't locate CSN 53c7ba75000400100000 in the changelog (DB rc=-30988). The consumer may need to be reinitialized.
[23/Jul/2014:12:52:13 +0100] NSMMReplicationPlugin - agmt="cn=meTorecsds4.bskyb.com" (recsds4:389): changelog iteration code returned a dummy entry with csn 53c7c6b1000200100000, skipping ...
[23/Jul/2014:12:52:13 +0100] agmt="cn=meTorecsds2.bskyb.com" (recsds2:389) - Can't locate CSN 53c7ba27000000100000 in the changelog (DB rc=-30988). The consumer may need to be reinitialized.

[root at recsds4 ~]# tail -f /var/log/dirsrv/slapd-RECS-BSKYB-COM/errors
[23/Jul/2014:12:52:03 +0100] ldbm_back_modify - Attempt to modify a tombstone entry nsuniqueid=b0838195-0da911e4-9433f833-313b8581,krbprincipalname=DNS/recsds1.bskyb.com at RECS.BSKYB.COM,cn=services,cn=accounts,dc=recs,dc=bskyb,dc=com
[23/Jul/2014:12:52:03 +0100] ldbm_back_modify - Attempt to modify a tombstone entry nsuniqueid=85992d8b-0da911e4-9433f833-313b8581,fqdn=recsds1.bskyb.com,cn=computers,cn=accounts,dc=recs,dc=bskyb,dc=com
[23/Jul/2014:12:52:06 +0100] ldbm_back_modify - Attempt to modify a tombstone entry nsuniqueid=b0838195-0da911e4-9433f833-313b8581,krbprincipalname=DNS/recsds1.bskyb.com at RECS.BSKYB.COM,cn=services,cn=accounts,dc=recs,dc=bskyb,dc=com

[root at recsds5 sch32]# tail -f /var/log/dirsrv/slapd-RECS-BSKYB-COM/errors
[23/Jul/2014:12:52:08 +0100] NSMMReplicationPlugin - agmt="cn=meTorecsds4.bskyb.com" (recsds4:389): Consumer failed to replay change (uniqueid 85992d8b-0da911e4-9433f833-313b8581, CSN 53c7ba7e000300100000): Server is unwilling to perform (53). Will retry later.
[23/Jul/2014:12:52:08 +0100] NSMMReplicationPlugin - agmt="cn=meTorecsds4.bskyb.com" (recsds4:389): Consumer failed to replay change (uniqueid b0838197-0da911e4-9433f833-313b8581, CSN 53c7ba90000000100000): Server is unwilling to perform (53). Will retry later.
[23/Jul/2014:12:52:16 +0100] NSMMReplicationPlugin - agmt="cn=meTorecsds4.bskyb.com" (recsds4:389): Consumer failed to replay change (uniqueid b0838195-0da911e4-9433f833-313b8581, CSN 53c7ba75000500100000): Server is unwilling to perform (53). Will retry later.

The background to this is a storage crash caused the master CA IAP to get fudged, and I then proceeded to promote a replica to master CA, re-added crashed IPAs and trying to sync them all up again and clean old orphaned RUVs.

Regards,
Suhail Choudhury.
DevOps | Recommendations Team | BSkyB


________________________________________
From: Martin Kosek [mkosek at redhat.com]
Sent: 23 July 2014 12:43
To: Choudhury, Suhail; freeipa-users at redhat.com
Subject: Re: [Freeipa-users] IPA Replication Status

On 07/23/2014 01:36 PM, Choudhury, Suhail wrote:
> Hi,
>
> I'm finding that on all IPA servers in 1 cluster the replication status shows as either "busy" or "started", but no "succeeded" status is being reported:
>
> [root at recsds2 ~]# ipa-replica-manage list -v $HOSTNAME
> recsds1.bskyb.com: replica
>   last init status: None
>   last init ended: None
>   last update status: 1 Can't acquire busy replica
>   last update ended: 2014-07-23 11:29:48+00:00
> recsds3.bskyb.com: replica
>   last init status: None
>   last init ended: None
>   last update status: 1 Can't acquire busy replica
>   last update ended: 2014-07-23 11:29:46+00:00
> [root at recsds2 ~]# ipa-replica-manage list -v $HOSTNAME
> recsds1.bskyb.com: replica
>   last init status: None
>   last init ended: None
>   last update status: 0 Replica acquired successfully: Incremental update started
>   last update ended: None
> recsds3.bskyb.com: replica
>   last init status: None
>   last init ended: None
>   last update status: 0 Replica acquired successfully: Incremental update started
>   last update ended: None
>
> This is as opposed to another IPA cluster:
>
> /home/sch # ipa-replica-manage list -v $HOSTNAME
> ipa01.ath.skycdc.com: replica
>   last init status: None
>   last init ended: None
>   last update status: 0 Replica acquired successfully: Incremental update succeeded
>   last update ended: 2014-07-23 11:24:22+00:00
> ipa01.hhe.skycdc.com: replica
>   last init status: None
>   last init ended: None
>   last update status: 0 Replica acquired successfully: Incremental update succeeded
>   last update ended: 2014-07-23 11:24:22+00:00
> ipa02.ath.skycdc.com: replica
>   last init status: None
>   last init ended: None
>   last update status: 0 Replica acquired successfully: Incremental update succeeded
>   last update ended: 2014-07-23 11:24:22+00:00
> ipa02.hhe.skycdc.com: replica
>   last init status: None
>   last init ended: None
>   last update status: 0 Replica acquired successfully: Incremental update succeeded
>   last update ended: 2014-07-23 11:24:22+00:00
> ipa02.slu.skycdc.com: replica
>   last init status: None
>   last init ended: None
>   last update status: 0 Replica acquired successfully: Incremental update succeeded
>   last update ended: 2014-07-23 11:24:21+00:00
>
> Do you know what may be causing this?

It is difficult to advise with just this information, you would need to check
for errors in DS log (/var/log/dirsrv/slapd-YOUR-REALM/errors).

Martin
Information in this email including any attachments may be privileged, confidential and is intended exclusively for the addressee. The views expressed may not be official policy, but the personal views of the originator. If you have received it in error, please notify the sender by return e-mail and delete it from your system. You should not reproduce, distribute, store, retransmit, use or disclose its contents to anyone. Please note we reserve the right to monitor all e-mail communication through our internal and external networks. SKY and the SKY marks are trademarks of British Sky Broadcasting Group plc and Sky International AG and are used under licence. British Sky Broadcasting Limited (Registration No. 2906991), Sky-In-Home Service Limited (Registration No. 2067075) and Sky Subscribers Services Limited (Registration No. 2340150) are direct or indirect subsidiaries of British Sky Broadcasting Group plc (Registration No. 2247735). All of the companies mentioned in this paragraph are incorporated in England and Wales and share the same registered office at Grant Way, Isleworth, Middlesex TW7 5QD.




More information about the Freeipa-users mailing list