[Freeipa-users] Replica cannot be reinitialized after upgrade

[Maciej Drobniuch]

Hi Goran

Exact same issue here with the same troubleshooting steps taken(I've tried
to reinitialize the replicas with success msg) - no luck so far.

I've additionally have run ipa_check_consistency script:
FreeIPA servers:    ipa1      ipa2      ipa3    STATE
===================================================================
Active Users        37            37            37            OK
Stage Users         0             0             0             OK
Preserved Users     0             0             0             OK
User Groups         10            10            10            OK
Hosts               69            69            69            OK
Host Groups         7             7             7             OK
HBAC Rules          11            11            11            OK
SUDO Rules          1             1             1             OK
DNS Zones           8             8             8             OK
LDAP Conflicts      YES           YES           YES           FAIL
Ghost Replicas      NO            NO            NO            OK
Anonymous BIND      YES           YES           YES           OK
Replication Status  ipa2 18   ipa1 0    ipa1 0
                    ipa3 0
===================================================================

Besides of this the ipa master named-pkcs is sometimes crashing and ipa
fails to start.
I've rolled a backup from 1week ago and it's starting but I don't know how
long it will last.

IPA team please help.


# ipa --version
VERSION: 4.4.0, API_VERSION: 2.213

-- 
Best regards

Maciej Drobniuch
Network Security Engineer
Collective-Sense,LLC


On Thu, May 11, 2017 at 6:53 PM, Goran Marik <goranm at ecobee.com> wrote:

> Hi,
>
> After an upgrade to Centos 7.3.1611 with “yum update", we started seeing
> the following messages in the logs:
> “””
> May  9 21:58:28 inf01 ns-slapd[4323]: [09/May/2017:21:58:28.519724479
> +0000] NSMMReplicationPlugin - changelog program - agmt="cn=cloneAgreement1-
> inf02.dev.ecobee.com-pki-tomcat" (inf02:389): CSN 576b34e8000a050f0000
> not found, we aren't as up to date, or we purged
> May  9 21:58:28 inf01 ns-slapd[4323]: [09/May/2017:21:58:28.550459233
> +0000] NSMMReplicationPlugin - agmt="cn=cloneAgreement1-
> inf02.dev.ecobee.com-pki-tomcat" (inf02:389): Data required to update
> replica has been purged from the changelog. The replica must be
> reinitialized.
> May  9 21:58:32 inf01 ns-slapd[4323]: [09/May/2017:21:58:32.588245476
> +0000] agmt="cn=cloneAgreement1-inf02.dev.ecobee.com-pki-tomcat"
> (inf02:389) - Can't locate CSN 576b34e8000a050f0000 in the changelog (DB
> rc=-30988). If replication stops, the consumer may need to be reinitialized.
> May  9 21:58:32 inf01 ns-slapd[4323]: [09/May/2017:21:58:32.611400689
> +0000] NSMMReplicationPlugin - changelog program - agmt="cn=cloneAgreement1-
> inf02.dev.ecobee.com-pki-tomcat" (inf02:389): CSN 576b34e8000a050f0000
> not found, we aren't as up to date, or we purged
> May  9 21:58:32 inf01 ns-slapd[4323]: [09/May/2017:21:58:32.642226385
> +0000] NSMMReplicationPlugin - agmt="cn=cloneAgreement1-
> inf02.dev.ecobee.com-pki-tomcat" (inf02:389): Data required to update
> replica has been purged from the changelog. The replica must be
> reinitialized.
> “””
>
> The log messages are pretty frequently, every few seconds, and report few
> different CSN numbers that cannot be located.
>
> This happens only on one replica out of 4. We’ve tried "ipa-replica-manage
> re-initialize —from” and “ipa-csreplica-manage re-initialize —from” several
> times, but while both commands report success, the log messages continue to
> happen. The server was rebooted and “systemctl restart ipa” was done few
> times as well.
>
> The replica seems to be working fine despite the errors, but I’m worried
> that the logs indicate underlaying problem we are not fully detecting. I
> would like to understand better what is triggering this behaviour and how
> to fix it, and if someone else saw them after a recent upgrades.
>
> The software versions are 389-ds-base-1.3.5.10-20.el7_3.x86_64 and
> ipa-server-4.4.0-14.el7.centos.7.x86_64
>
> Thanks,
> Goran
>
> --
> Goran Marik
> Senior Systems Developer
>
> ecobee
> 250 University Ave, Suite 400
> Toronto, ON M5H 3E5
>
>
>
> --
> Manage your subscription for the Freeipa-users mailing list:
> https://www.redhat.com/mailman/listinfo/freeipa-users
> Go to http://freeipa.org for more info on the project
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/freeipa-users/attachments/20170515/3721c0cb/attachment.htm>