[Freeipa-users] Incremental update failed and requires administrator action

[bahan w]

Hello !

I recently installed a replica (master2) in addition of my master (master1)
with IPA 3.0.0-47 on RHEL6.6.
I don't know from when exactly, but the dirsrv (and the whole ipa service)
on master1 crashes regularly with the following logs.

###
[22/Jan/2016:15:38:20 +0100] - 389-Directory/1.2.11.15 B2015.279.183
starting up
[22/Jan/2016:15:38:20 +0100] schema-compat-plugin - warning: no entries set
up under cn=computers, cn=compat,dc=<myrealm>
[22/Jan/2016:15:38:21 +0100] schema-compat-plugin - warning: no entries set
up under cn=ng, cn=compat,dc=<myrealm>
[22/Jan/2016:15:38:21 +0100] schema-compat-plugin - warning: no entries set
up under ou=sudoers,dc=<myrealm>
[22/Jan/2016:15:38:21 +0100] - slapd started.  Listening on All Interfaces
port 389 for LDAP requests
[22/Jan/2016:15:38:21 +0100] - Listening on All Interfaces port 636 for
LDAPS requests
[22/Jan/2016:15:38:21 +0100] - Listening on /var/run/slapd-<myrealm>.socket
for LDAPI requests
[22/Jan/2016:17:04:03 +0100] NSMMReplicationPlugin - changelog program -
_cl5WriteOperationTxn: retry (49) the transaction
(csn=56a252ef000000040000) failed (rc=-30994 (DB_LOCK_DEADLOCK: Locker
killed to resolve a deadlock))
[22/Jan/2016:17:04:03 +0100] NSMMReplicationPlugin - changelog program -
_cl5WriteOperationTxn: failed to write entry with csn
(56a252ef000000040000); db error - -30994 DB_LOCK_DEADLOCK: Locker killed
to resolve a deadlock
[22/Jan/2016:17:04:03 +0100] NSMMReplicationPlugin -
write_changelog_and_ruv: can't add a change for
uid=<user1>,cn=users,cn=accounts,dc=<myrealm> (uniqid:
a7ebd403-c12111e5-9c84c092-9a5deb81, optype: 16) to changelog csn
56a252ef000000040000
[22/Jan/2016:17:04:03 +0100] NSMMReplicationPlugin -
agmt="cn=meTo<master2>" (<shortname_master2>:389): Missing data encountered
[22/Jan/2016:17:04:03 +0100] NSMMReplicationPlugin -
agmt="cn=meTo<master2>" (<shortname_master2>:389): Incremental update
failed and requires administrator action
###

Then the dirsrv, I mean the whole ipa server, is down.
When I restart the service, here is what is see :

###
[22/Jan/2016:17:06:18 +0100] - 389-Directory/1.2.11.15 B2015.279.183
starting up
[22/Jan/2016:17:06:18 +0100] - Detected Disorderly Shutdown last time
Directory Server was running, recovering database.
[22/Jan/2016:17:06:18 +0100] schema-compat-plugin - warning: no entries set
up under cn=computers, cn=compat,dc=<myrealm>
[22/Jan/2016:17:06:19 +0100] schema-compat-plugin - warning: no entries set
up under cn=ng, cn=compat,dc=<myrealm>
[22/Jan/2016:17:06:19 +0100] schema-compat-plugin - warning: no entries set
up under ou=sudoers,dc=<myrealm>
[22/Jan/2016:17:06:20 +0100] set_krb5_creds - Could not get initial
credentials for principal [ldap/<master1>@<myrealm>] in keytab
[FILE:/etc/dirsrv/ds.keytab]: -1765328324 (Generic error (see e-text))
[22/Jan/2016:17:06:20 +0100] - slapd started.  Listening on All Interfaces
port 389 for LDAP requests
[22/Jan/2016:17:06:20 +0100] - Listening on All Interfaces port 636 for
LDAPS requests
[22/Jan/2016:17:06:20 +0100] - Listening on /var/run/slapd-<myrealm>.socket
for LDAPI requests
[22/Jan/2016:17:06:20 +0100] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure.  Minor code may provide more information (Credentials cache file
'/tmp/krb5cc_244' not found)) errno 0 (Success)
[22/Jan/2016:17:06:20 +0100] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[22/Jan/2016:17:06:20 +0100] NSMMReplicationPlugin -
agmt="cn=meTo<master2>" (<shortname_master2>:389): Replication bind with
GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic failure:
GSSAPI Error: Unspecified GSS failure.  Minor code may provide more
information (Credentials cache file '/tmp/krb5cc_244' not found))
[22/Jan/2016:17:06:23 +0100] NSMMReplicationPlugin -
agmt="cn=meTo<master2>" (<shortname_master2>:389): Replication bind with
GSSAPI auth resumed
###

It seems that there is a problem to write an entry in the DB ? Do you know
how I can solve this problem please ?

Furthermore, it seems that there is a second problem with the keytab
/etc/dirsrv/ds.keytab.

The keytab is good for me :
###
#ls -l /etc/dirsrv/ds.keytab
-rw------- 1 dirsrv dirsrv 362 Jan 21 14:12 /etc/dirsrv/ds.keytab
# kinit -kt /etc/dirsrv/ds.keytab ldap/<master1>@<myrealm>
# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: ldap/<master1>@<myrealm>

Valid starting     Expires            Service principal
01/25/16 11:54:23  01/26/16 11:54:23  krbtgt/<myrealm>@<myrealm>
###

I wonder if this second problem does not come from the user dirsrv who
would not be able to use this keytab.
I cannot test this because this user dirsrv has been created with nologin.
###
# su - dirsrv -c "kinit -kt /etc/dirsrv/ds.keytab ldap/<master1>@<myrealm>"
This account is currently not available.

# grep dirsrv /etc/passwd
dirsrv:x:244:497::/var/lib/dirsrv:/sbin/nologin
pkisrv:x:246:497::/var/lib/dirsrv:/sbin/nologin
###

Just for my information, is it normal that these users are created with
nologin ?

Best regards.

Bahan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/freeipa-users/attachments/20160125/616ccff2/attachment.htm>