[Freeipa-devel] Database error on replicas

Fri Jun 5 17:33:23 UTC 2015

Hi,

So far I am still unable to reproduce the problem.
Comparing the errors logs of failing replica vs successful replica they 
are very similar. Except this failure

Failing one

    ...
    [03/Jun/2015:03:45:33 -0400] slapd_ldap_sasl_interactive_bind -
    Error: could not perform interactive bind for id [] mech [GSSAPI]:
    *LDAP error -1 (Can't contact LDAP server)* ((null)) errno 115
    (Operation now in progress)
    [03/Jun/2015:03:45:33 -0400] slapi_ldap_bind - Error: could not
    perform interactive bind for id [] authentication mechanism
    [GSSAPI]: error -1 (Can't contact LDAP server)
    [03/Jun/2015:03:45:33 -0400] NSMMReplicationPlugin -
    agmt="cn=meTotestmaster.zaeba.li" (testmaster:389): Replication bind
    with GSSAPI auth failed: LDAP error -1 (Can't contact LDAP server) ()
    [03/Jun/2015:03:45:38 -0400] slapd_ldap_sasl_interactive_bind -
    Error: could not perform interactive bind for id [] mech [GSSAPI]:
    LDAP error -1 (Can't contact LDAP server) ((null)) errno 2 (No such
    file or directory)
    <many errors>
    ...

Successful one:

    ...
    [05/Jun/2015:17:51:20 +0200] NSMMReplicationPlugin -
    agmt="cn=meTovm-229.idm.lab.eng.brq.redhat.com" (vm-229:389):
    Replication bind with GSSAPI auth failed: *LDAP error -2 (Local
    error)* (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
    failure.  Minor code may provide more information (No Kerberos
    credentials available))
    [05/Jun/2015:17:51:23 +0200] NSMMReplicationPlugin -
    agmt="cn=meTovm-229.idm.lab.eng.brq.redhat.com" (vm-229:389):
    Replication bind with GSSAPI auth resumed
    [05/Jun/2015:18:47:26 +0200] - slapd shutting down - signaling
    operation threads - op stack size 7 max work q size 2 max work q
    stack size 2
    [05/Jun/2015:18:47:26 +0200] - slapd shutting down - waiting for 1
    thread to terminate
    [05/Jun/2015:18:47:26 +0200] - slapd shutting down - closing down
    internal subsystems and plugins
    [05/Jun/2015:18:47:26 +0200] - Waiting for 4 database threads to stop
    [05/Jun/2015:18:47:27 +0200] - All database threads now stopped
    [05/Jun/2015:18:47:27 +0200] - slapd shutting down - freed 2 work q
    stack objects - freed 8 op stack objects
    [05/Jun/2015:18:47:27 +0200] - slapd stopped.
    ...

This is looking like in the failing case, the replica is not able to 
connect to the master.
In the successful tests I did not install DNS while it was installed in 
the failing tests.
We need to retry with DNS configuration, because it could be part of the 
failure to access the master host.

thanks
theirry

On 06/04/2015 07:27 PM, thierry bordaz wrote:
> Hello Oleg,
>
> So far I have been unable to reproduce the problem.
> I tried various scenarios depending if the first update was on 
> master/slave, or with 2 slaves, 1 slave, 1slave added later.
>
> Do you have any detail how you did your test ?
>
> If you can restart the remaining VM, I would be interested in the logs 
> (access/errors).
>
> thanks
> thierry
> On 06/03/2015 11:11 AM, Oleg Fayans wrote:
>> Hi Martin,
>>
>> On 06/03/2015 10:46 AM, Martin Babinsky wrote:
>>> On 06/03/2015 10:33 AM, Oleg Fayans wrote:
>>>> Hi,
>>>>
>>>> With the latest freeipa code containing Topology plugin patches, I am
>>>> unable to make any changes in replicas.
>>>>
>>>> I have the following topology:
>>>> replica1 <=> master <=> replica3
>>>> Here is the output of the ipa topologysegment-find command:
>>>>
>>>> Suffix name: realm
>>>> ------------------
>>>> 2 segments matched
>>>> ------------------
>>>>    Segment name: replica1.zaeba.li-to-testmaster.zaeba.li
>>>>    Left node: replica1.zaeba.li
>>>>    Right node: testmaster.zaeba.li
>>>>    Connectivity: both
>>>>
>>>>    Segment name: replica3.zaeba.li-to-testmaster.zaeba.li
>>>>    Left node: replica3.zaeba.li
>>>>    Right node: testmaster.zaeba.li
>>>>    Connectivity: both
>>>> ----------------------------
>>>> Number of entries returned 2
>>>> ----------------------------
>>>>
>>>>
>>>> Any changes on master get replicated to replicas successfully. 
>>>> However,
>>>> any attempts to change anything on replicas, for example, create a 
>>>> user,
>>>> result in the error message about DatabaseError (attached).
>>>>
>>>> The corresponding part of the dirsrv log looks like this:
>>>>
>>>> 03/Jun/2015:04:11:55 -0400] slapi_ldap_bind - Error: could not perform
>>>> interactive bind for id [] authentication mechanism [GSSAPI]: error -1
>>>> (Can't contact LDAP server)
>>>> [03/Jun/2015:04:15:02 -0400] slapi_ldap_bind - Error: could not send
>>>> startTLS request: error -1 (Can't contact LDAP server) errno 0 
>>>> (Success)
>>>> [03/Jun/2015:04:16:55 -0400] slapd_ldap_sasl_interactive_bind - Error:
>>>> could not perform interactive bind for id [] mech [GSSAPI]: LDAP error
>>>> -1 (Can't contact LDAP server) ((null)) errno 2 (No such file or 
>>>> directory)
>>>> [03/Jun/2015:04:16:55 -0400] slapi_ldap_bind - Error: could not 
>>>> perform
>>>> interactive bind for id [] authentication mechanism [GSSAPI]: error -1
>>>> (Can't contact LDAP server)
>>>>
>>>> The full log is attached
>>>>
>>>>
>>>>
>>> Hi Oleg,
>>>
>>> could you also post the output of 'journalctl -xe' related to dirsrv 
>>> (on master and also on replicas)? I have seen a couple of segfaults 
>>> there during reviewing Petr Vobornik's topology* commands.
>>>
>> Attached
>>
>>
>>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20150605/c86e3833/attachment.htm>