[Freeipa-users] more replication issues

[Rich Megginson]

On 05/15/2015 09:53 AM, Janelle wrote:
>> On May 15, 2015, at 08:57, Ludwig Krispenz <lkrispen at redhat.com> wrote:
>>
>>
>>> On 05/15/2015 02:45 PM, Janelle wrote:
>>>> On 5/15/15 3:30 AM, Ludwig Krispenz wrote:
>>>>
>>>>> On 05/13/2015 06:34 PM, Janelle wrote:
>>>>>> On 5/13/15 9:13 AM, Rich Megginson wrote:
>>>>>>> On 05/13/2015 10:04 AM, Janelle wrote:
>>>>>>>> On 5/13/15 8:49 AM, Rich Megginson wrote:
>>>>>>>>> On 05/13/2015 09:40 AM, Janelle wrote:
>>>>>>>>> Recently I started seeing these crop up across my servers:
>>>>>>>>>
>>>>>>>>> slapi_ldap_bind - Error: could not bind id [cn=Replication Manager masterAgreement1-ipa01.example.com-pki-tomcat,ou=csusers,cn=config] authentication mechanism [SIMPLE]: error 32 (No such object) errno 0 (Success)
>>>>>>>> Does that entry exist?
>>>>>>>>
>>>>>>>> ldapsearch -xLLL -h consumer.host -D "cn=directory manager" -W -s base -b "cn=Replication Manager masterAgreement1-ipa01.example.com-pki-tomcat,ou=csusers,cn=config"
>>>>>>>>
>>>>>>>> Does the parent exist?
>>>>>>>>
>>>>>>>> ldapsearch -xLLL -h consumer.host -D "cn=directory manager" -W -s base -b "ou=csusers,cn=config"
>>>>>>> I am finding that there does seem to be a relation to the above error and a possible CSN issue:
>>>>>>>
>>>>>>> Can't locate CSN 555131e5000200190000 in the changelog (DB rc=-30988). If replication stops, the consumer may need to be reinitialized.
>>>>>>>
>>>>>>> I guess what concerns me is what could be causing this. We don't do a lot of changes all the time.
>>>>>>>
>>>>>>> And in answer to the question above - we seem to have last the agreement somehow:
>>>>>>>
>>>>>>> No such object (32)
>>>>>>>
>>>>>> Is there a DEL operation in the access log for "cn=Replication Manager masterAgreement1-ipa01.example.com-pki-tomcat,ou=csusers,cn=config"?
>>>>>>
>>>>>> maybe something like
>>>>>>
>>>>>> # grep DEL /var/log/dirsrv/slapd-INST/access|grep -i "Replication Manager"
>>>>>>
>>>>> nope -- none of the servers have it.
>>>> your original message is very clear:
>>>>
>>>> could not bind id [cn=Replication Manager masterAgreement1-ipa01.example.com-pki-tomcat,ou=csusers,cn=config] authentication mechanism [SIMPLE]: error 32 (No such object) errno 0 (Success)
>>>>
>>>> this means that you have replication agreement wth SIMPLE auth which uses a
>>>> nsDS5ReplicaBindDN: cn=Replication Manager masterAgreement1-ipa01.example.com-pki-tomcat,ou=csusers,cn=config
>>>>
>>>> which does not exist on the target server of the agreement. Now you say it was never deleted, so it was probably never added, but used in the replication agreements. How do you manage and setup replication agreements ?
>>>>
>>> All replicas are configred simply:
>>>
>>> ipa-replica-prepare hostname...
>>> scp ..
>>> ipa-replica-install --no-ntp --setup-ca Replica-file
>>>
>>> That is it. NTP is not set because internal NTP servers are used. All replicas are CA replicas for safety (no certs are managed)
>> ok, I was a bit puzzled because ipa uses ldapprincipals and gssapi for the main suffix replication.
>> But I just verified that after ipa-replica-install --setup-ca CA replication is setup with users in ou=csusers,cn=config and uses it as replica binddn, I have no idea why it would disappear.
>>
>> when Rich asked to search for a DEL, did you check this on the server that logged the message or on the endpoint of the replication agreement (it should be there), and you may have to check in the rotated access logs access.<timestamp> as well
> Checked it on ALL servers just to be sure.
>
> ~J
>

If it is present at some point, then is missing, it must be some 
internal operation that is removing it.  Please enable access logging of 
internal operations:

ldapmodify -x -h consumer.host -D "cn=directory manager" -w "password" <<EOF
dn: cn=config
changetype: modify
replace: nsslapd-accesslog-level
nsslapd-accesslog-level: 4
EOF

Then you will have to wait until the problem reoccurs

Is or was the server ipa01.example.com the target of a host delete, 
replica delete, or cleanallruv operation?