[Freeipa-users] IPA 3.0.47 to 3.0.50 Upgrade problem
Sean Hogan
schogan at us.ibm.com
Mon Jun 20 18:36:25 UTC 2016
Hi All..
I thought we fixed this issue by rebooting the KVM host but it is showing
again. Our First Master IPA is being rebooted 2 -5 times a day now just to
keep it alive.
What we are seeing:
God at FirstMaster log]# kinit admin
kinit: Cannot contact any KDC for realm 'Domain.LOCAL' while getting
initial credentials
DNS is not working as nslookup is failing to a replica.... think once we
lose DNS it all goes down hill which makes sense.
[god at FirstMaster log]# ipactl stop -----> Just hangs forever.. no
replies.. no error.. nothing
I try service named stop and nothing happens
I have the box hard shutdown from KVM console. Reboot it and it works for
a little while but eventually back to same behavior.
At this point I can service named stop and it responds... ipactl status and
it responds.. but when if I try service named restart I get
[god at FirstMaster log]# service named stop
Stopping named: ......
[god at Firstmaster log]# service named start
Starting named: [FAILED]
[god at FirstMaster log]# service named status
rndc: connect failed: 127.0.0.1#953: connection refused
named dead but pid file exists
Rebooted box and it is hung on shutting down domain-local and never fully
shuts down.. have to get it hard shutdown again.
During an attempt to gracefully shut down we see this
Shutting Down dirsrv:
PKI-IPA OK
DOMAIN-LOCAL FAILED
*** Error: 1 instance(s) unsuccessfully stopped FAILED
Then it moves on to shut other things down and returns to dirsrv
Shutting Down dirsrv:
PKI-IPA....server already stopped
FAILED {Makes sense.. it died earlier}
DOMAIN-LOCAL...
{this sits here til we hard shutdown}
bind-libs-9.8.2-0.47.rc1.el6.x86_64
bind-9.8.2-0.47.rc1.el6.x86_64
bind-utils-9.8.2-0.47.rc1.el6.x86_64
ipa-client-3.0.0-50.el6.1.x86_64
ipa-server-selinux-3.0.0-50.el6.1.x86_64
ipa-server-3.0.0-50.el6.1.x86_64
sssd-ipa-1.13.3-22.el6.x86_64
/var/log/dirsrv/slapd-DOMAIN-LOCAL
[20/Jun/2016:13:29:06 -0400] - 389-Directory/1.2.11.15 B2016.063.2110
starting up
[20/Jun/2016:13:29:06 -0400] schema-compat-plugin - warning: no entries set
up under cn=computers, cn=compat,dc=domain,dc=local
[20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin - ruv_compare_ruv: RUV
[database RUV] does not contain element [{replica 7} 55ca26a0000900070000
5688d8e6001000070000] which is present in RUV [changelog max RUV]
[20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin -
replica_check_for_data_reload: Warning: for replica dc=domain,dc=local
there were some differences between the changelog max RUV and the database
RUV. If there are obsolete elements in the database RUV, you should remove
them using the CLEANALLRUV task. If they are not obsolete, you should
check their status to see why there are no changes from those servers in
the changelog.
[20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial
credentials for principal [ldap/server1.domain.local at DOMAIN.LOCAL] in
keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
for requested realm)
[20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial
credentials for principal [ldap/server1.domain.local at DOMAIN.LOCAL] in
keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
for requested realm)
[20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial
credentials for principal [ldap/server1.domain.local at DOMAIN.LOCAL] in
keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
for requested realm)
[20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (Credentials cache file
'/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
[20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver8.domain.local" (server8:389): Replication bind with
GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic failure:
GSSAPI Error: Unspecified GSS failure. Minor code may provide more
information (Credentials cache file '/tmp/krb5cc_495' not found))
[20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial
credentials for principal [ldap/server1.domain.local at DOMAIN.LOCAL] in
keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
for requested realm)
[20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (Credentials cache file
'/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
[20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver9.domain.local" (server9:389): Replication bind with
GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic failure:
GSSAPI Error: Unspecified GSS failure. Minor code may provide more
information (Credentials cache file '/tmp/krb5cc_495' not found))
[20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial
credentials for principal [ldap/server1.domain.local at DOMAIN.LOCAL] in
keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
for requested realm)
[20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (Credentials cache file
'/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
[20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver3.domain.local" (server3:389): Replication bind with
GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic failure:
GSSAPI Error: Unspecified GSS failure. Minor code may provide more
information (Credentials cache file '/tmp/krb5cc_495' not found))
[20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial
credentials for principal [ldap/server1.domain.local at DOMAIN.LOCAL] in
keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
for requested realm)
[20/Jun/2016:13:29:07 -0400] - slapd started. Listening on All Interfaces
port 389 for LDAP requests
[20/Jun/2016:13:29:07 -0400] - Listening on All Interfaces port 636 for
LDAPS requests
[20/Jun/2016:13:29:07 -0400] - Listening
on /var/run/slapd-DOMAIN-LOCAL.socket for LDAPI requests
[20/Jun/2016:13:29:07 -0400] set_krb5_creds - Could not get initial
credentials for principal [ldap/server1.domain.local at DOMAIN.LOCAL] in
keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
for requested realm)
[20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (Credentials cache file
'/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
[20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver4.domain.local" (server4:389): Replication bind with
GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic failure:
GSSAPI Error: Unspecified GSS failure. Minor code may provide more
information (Credentials cache file '/tmp/krb5cc_495' not found))
[20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (Credentials cache file
'/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
[20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin -
agmt="cn=meTo1server.domain.local" (1server:389): Replication bind with
GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic failure:
GSSAPI Error: Unspecified GSS failure. Minor code may provide more
information (Credentials cache file '/tmp/krb5cc_495' not found))
[20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (Credentials cache file
'/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
[20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver7.domain.local" (server7:389): Replication bind with
GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic failure:
GSSAPI Error: Unspecified GSS failure. Minor code may provide more
information (Credentials cache file '/tmp/krb5cc_495' not found))
[20/Jun/2016:13:29:07 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (Credentials cache file
'/tmp/krb5cc_495' not found)) errno 0 (Success)
[20/Jun/2016:13:29:07 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:29:07 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver2.domain.local" (server2:389): Replication bind with
GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic failure:
GSSAPI Error: Unspecified GSS failure. Minor code may provide more
information (Credentials cache file '/tmp/krb5cc_495' not found))
[20/Jun/2016:13:29:10 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver9.domain.local" (server9:389): Replication bind with
GSSAPI auth resumed
[20/Jun/2016:13:29:10 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error 49
(Invalid credentials) (SASL(-13): authentication failure: GSSAPI Failure:
gss_accept_sec_context) errno 0 (Success)
[20/Jun/2016:13:29:10 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error 49 (Invalid credentials)
[20/Jun/2016:13:29:10 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver3.domain.local" (server3:389): Replication bind with
GSSAPI auth failed: LDAP error 49 (Invalid credentials) (SASL(-13):
authentication failure: GSSAPI Failure: gss_accept_sec_context)
[20/Jun/2016:13:29:10 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver2.domain.local" (server2:389): Replication bind with
GSSAPI auth resumed
[20/Jun/2016:13:29:10 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:29:10 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver8.domain.local" (server8:389): Replication bind with
GSSAPI auth resumed
[20/Jun/2016:13:29:10 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (No credentials cache
found)) errno 2 (No such file or directory)
[20/Jun/2016:13:29:10 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:29:10 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (No credentials cache
found)) errno 2 (No such file or directory)
[20/Jun/2016:13:29:10 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:29:16 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error 49
(Invalid credentials) (SASL(-13): authentication failure: GSSAPI Failure:
gss_accept_sec_context) errno 0 (Success)
[20/Jun/2016:13:29:16 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error 49 (Invalid credentials)
[20/Jun/2016:13:59:00 -0400] - 389-Directory/1.2.11.15 B2016.063.2110
starting up
[20/Jun/2016:13:59:00 -0400] - Detected Disorderly Shutdown last time
Directory Server was running, recovering database.
[20/Jun/2016:13:59:01 -0400] schema-compat-plugin - warning: no entries set
up under cn=computers, cn=compat,dc=domain,dc=local
[20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin - ruv_compare_ruv: RUV
[database RUV] does not contain element [{replica 7} 55ca26a0000900070000
5688d8e6001000070000] which is present in RUV [changelog max RUV]
[20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin -
replica_check_for_data_reload: Warning: for replica dc=domain,dc=local
there were some differences between the changelog max RUV and the database
RUV. If there are obsolete elements in the database RUV, you should remove
them using the CLEANALLRUV task. If they are not obsolete, you should
check their status to see why there are no changes from those servers in
the changelog.
[20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial
credentials for principal [ldap/server1.domain.local at DOMAIN.LOCAL] in
keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
for requested realm)
[20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial
credentials for principal [ldap/server1.domain.local at DOMAIN.LOCAL] in
keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
for requested realm)
[20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial
credentials for principal [ldap/server1.domain.local at DOMAIN.LOCAL] in
keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
for requested realm)
[20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (Credentials cache file
'/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
[20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (Credentials cache file
'/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
[20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver9.domain.local" (server9:389): Replication bind with
GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic failure:
GSSAPI Error: Unspecified GSS failure. Minor code may provide more
information (Credentials cache file '/tmp/krb5cc_495' not found))
[20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial
credentials for principal [ldap/server1.domain.local at DOMAIN.LOCAL] in
keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
for requested realm)
[20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver8.domain.local" (server8:389): Replication bind with
GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic failure:
GSSAPI Error: Unspecified GSS failure. Minor code may provide more
information (Credentials cache file '/tmp/krb5cc_495' not found))
[20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (Credentials cache file
'/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
[20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin -
agmt="cn=meTobldvxl0011.domain.local" (1server:389): Replication bind with
GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic failure:
GSSAPI Error: Unspecified GSS failure. Minor code may provide more
information (Credentials cache file '/tmp/krb5cc_495' not found))
[20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (Credentials cache file
'/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
[20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver3.domain.local" (server3:389): Replication bind with
GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic failure:
GSSAPI Error: Unspecified GSS failure. Minor code may provide more
information (Credentials cache file '/tmp/krb5cc_495' not found))
[20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial
credentials for principal [ldap/server1.domain.local at DOMAIN.LOCAL] in
keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
for requested realm)
[20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial
credentials for principal [ldap/server1.domain.local at DOMAIN.LOCAL] in
keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
for requested realm)
[20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (Credentials cache file
'/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
[20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver7.domain.local" (server7:389): Replication bind with
GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic failure:
GSSAPI Error: Unspecified GSS failure. Minor code may provide more
information (Credentials cache file '/tmp/krb5cc_495' not found))
[20/Jun/2016:13:59:48 -0400] - slapd started. Listening on All Interfaces
port 389 for LDAP requests
[20/Jun/2016:13:59:48 -0400] - Listening on All Interfaces port 636 for
LDAPS requests
[20/Jun/2016:13:59:48 -0400] - Listening
on /var/run/slapd-DOMAIN-LOCAL.socket for LDAPI requests
[20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (Credentials cache file
'/tmp/krb5cc_495' not found)) errno 2 (No such file or directory)
[20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver4.domain.local" (server4:389): Replication bind with
GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic failure:
GSSAPI Error: Unspecified GSS failure. Minor code may provide more
information (Credentials cache file '/tmp/krb5cc_495' not found))
[20/Jun/2016:13:59:48 -0400] set_krb5_creds - Could not get initial
credentials for principal [ldap/server1.domain.local at DOMAIN.LOCAL] in
keytab [FILE:/etc/dirsrv/ds.keytab]: -1765328228 (Cannot contact any KDC
for requested realm)
[20/Jun/2016:13:59:48 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (Credentials cache file
'/tmp/krb5cc_495' not found)) errno 0 (Success)
[20/Jun/2016:13:59:48 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:59:48 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver2.domain.local" (server2:389): Replication bind with
GSSAPI auth failed: LDAP error -2 (Local error) (SASL(-1): generic failure:
GSSAPI Error: Unspecified GSS failure. Minor code may provide more
information (Credentials cache file '/tmp/krb5cc_495' not found))
[20/Jun/2016:13:59:51 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver8.domain.local" (server8:389): Replication bind with
GSSAPI auth resumed
[20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error 49
(Invalid credentials) (SASL(-13): authentication failure: GSSAPI Failure:
gss_accept_sec_context) errno 0 (Success)
[20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error 49 (Invalid credentials)
[20/Jun/2016:13:59:51 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver3.domain.local" (server3:389): Replication bind with
GSSAPI auth failed: LDAP error 49 (Invalid credentials) (SASL(-13):
authentication failure: GSSAPI Failure: gss_accept_sec_context)
[20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (No credentials cache
found)) errno 2 (No such file or directory)
[20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (No credentials cache
found)) errno 2 (No such file or directory)
[20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (No credentials cache
found)) errno 2 (No such file or directory)
[20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (No credentials cache
found)) errno 2 (No such file or directory)
[20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:59:51 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (No credentials cache
found)) errno 2 (No such file or directory)
[20/Jun/2016:13:59:51 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[20/Jun/2016:13:59:57 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error 49
(Invalid credentials) (SASL(-13): authentication failure: GSSAPI Failure:
gss_accept_sec_context) errno 0 (Success)
[20/Jun/2016:13:59:57 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error 49 (Invalid credentials)
[20/Jun/2016:13:59:57 -0400] NSMMReplicationPlugin -
agmt="cn=meToserver2.domain.local" (server2:389): Replication bind with
GSSAPI auth resumed
Sean Hogan
From: Sean Hogan/Durham/IBM
To: freeipa-users <freeipa-users at redhat.com>
Date: 06/02/2016 09:24 AM
Subject: IPA 3.0.47 to 3.0.50 Upgrade problem
Hello All,
Recently went from RHEL 6.7 IPA 3.0.47 to 6.8 IPA 3.0.50. I also think
(not sure on this yet) that they changed ntp.. ntp used to point at my
ipas.. but they look like they are now pointing elsewhere. Everything was
stable at 6.7 3.0.47 pointing to IPA for NTP. However.. they all seem to
have the same date.
My master first IPA is acting up. Replication is off, kerberos seems to be
off, DNS is off and I think IPA in general on it is toast.
We do have 8 IPAs.. only FirstMaster is acting up it seems right now and
all either running on KVM or ESXI.
[God at FirstMasterIPA slapd-DOMAIN-LOCAL]# kinit admin
kinit: Generic error (see e-text) while getting initial credential
slapd-DOMAIN-LOCAL
[01/Jun/2016:18:25:43 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (Cannot contact any KDC
for realm 'DOMAIN.LOCAL')) errno 115 (Operation now in progress)
[01/Jun/2016:18:25:43 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[01/Jun/2016:18:25:48 -0400] NSMMReplicationPlugin -
agmt="cn=meToipaserv2.domain.local" (ipaserv2:389): Replication bind with
GSSAPI auth resumed
[01/Jun/2016:18:25:48 -0400] NSMMReplicationPlugin -
agmt="cn=meToipaserv3.domain.local" (ipaserv3:389): Replication bind with
GSSAPI auth resumed
[01/Jun/2016:18:25:48 -0400] NSMMReplicationPlugin -
agmt="cn=meToipaserv4.domain.local" (ipaserv4:389): Replication bind with
GSSAPI auth resumed
[01/Jun/2016:18:25:48 -0400] NSMMReplicationPlugin -
agmt="cn=meToipaserv5.domain.local" (ipaserv5:389): Replication bind with
GSSAPI auth resumed
[01/Jun/2016:18:28:04 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error 49
(Invalid credentials) (SASL(-13): authentication failure: GSSAPI Failure:
gss_accept_sec_context) errno 0 (Success)
[01/Jun/2016:18:28:04 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error 49 (Invalid credentials)
[01/Jun/2016:18:28:13 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (No credentials cache
found)) errno 2 (No such file or directory)
[01/Jun/2016:18:28:13 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[01/Jun/2016:18:33:03 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error 49
(Invalid credentials) (SASL(-13): authentication failure: GSSAPI Failure:
gss_accept_sec_context) errno 0 (Success)
[01/Jun/2016:18:33:03 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error 49 (Invalid credentials)
[01/Jun/2016:18:33:18 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error -2
(Local error) (SASL(-1): generic failure: GSSAPI Error: Unspecified GSS
failure. Minor code may provide more information (No credentials cache
found)) errno 2 (No such file or directory)
[01/Jun/2016:18:33:18 -0400] slapi_ldap_bind - Error: could not perform
interactive bind for id [] mech [GSSAPI]: error -2 (Local error)
[01/Jun/2016:18:38:03 -0400] slapd_ldap_sasl_interactive_bind - Error:
could not perform interactive bind for id [] mech [GSSAPI]: LDAP error 49
(Invalid credentials) (SASL(-13): authentication failure: GSSAPI Failure:
gss_accept_sec_context) errno 0 (Success)
[God at FirstMasterIPA slapd-DOMAIN-LOCAL]# ipa-replica-manage -v list
--------------> just hangs and never returns
[God at FirstMasterIPA slapd-DOMAIN-LOCAL]# ipactl start ------------->Just
hangs here as well.. never gets to the KDC.
Starting Directory Service
Starting dirsrv:
PKI-IPA... already running [ OK ]
DOMAIN-LOCAL... already running [ OK ]
If I run nslookup it fails over to a Replica for the DNS resolution instead
of resolving ips itself.
PKI log shows a bunch of this:
[02/Jun/2016:11:15:25 -0400] NSMMReplicationPlugin -
agmt="cn=masterAgreement1-ipaserver2.domain.local-pki-ca" (ipaserver2:7389):
Replication bind with SIMPLE auth failed: LDAP error -1 (Can't contact
LDAP server) ((null))
[02/Jun/2016:11:15:34 -0400] NSMMReplicationPlugin -
agmt="cn=masterAgreement1-ipaserver2.domain.local-pki-ca" (ipaserver2:7389):
Replication bind with SIMPLE auth resumed
[02/Jun/2016:11:16:36 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 0 (Success)
[02/Jun/2016:11:16:51 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 107 (Transport
endpoint is not connected)
[02/Jun/2016:11:21:51 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 0 (Success)
[02/Jun/2016:11:22:06 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 107 (Transport
endpoint is not connected)
[02/Jun/2016:11:26:36 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 0 (Success)
[02/Jun/2016:11:26:41 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 107 (Transport
endpoint is not connected)
[02/Jun/2016:11:31:36 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 0 (Success)
[02/Jun/2016:11:31:41 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 107 (Transport
endpoint is not connected)
[02/Jun/2016:11:36:36 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 0 (Success)
[02/Jun/2016:11:36:41 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 107 (Transport
endpoint is not connected)
[02/Jun/2016:11:41:46 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 0 (Success)
[02/Jun/2016:11:41:51 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 107 (Transport
endpoint is not connected)
[02/Jun/2016:11:45:16 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 0 (Success)
[02/Jun/2016:11:45:16 -0400] NSMMReplicationPlugin -
agmt="cn=masterAgreement1-ipaserver3.domain.local-pki-ca" (ipaserver3:7389):
Replication bind with SIMPLE auth failed: LDAP error -1 (Can't contact
LDAP server) ((null))
[02/Jun/2016:11:45:25 -0400] NSMMReplicationPlugin -
agmt="cn=masterAgreement1-ipaserver3.domain.local-pki-ca" (ipaserver3:7389):
Replication bind with SIMPLE auth resumed
[02/Jun/2016:11:46:51 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 0 (Success)
[02/Jun/2016:11:46:56 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 107 (Transport
endpoint is not connected)
[02/Jun/2016:11:51:36 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 0 (Success)
[02/Jun/2016:11:51:41 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 107 (Transport
endpoint is not connected)
[02/Jun/2016:11:56:46 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 0 (Success)
[02/Jun/2016:11:56:51 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 107 (Transport
endpoint is not connected)
[02/Jun/2016:12:01:36 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 0 (Success)
[02/Jun/2016:12:01:41 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 107 (Transport
endpoint is not connected)
[02/Jun/2016:12:05:33 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 0 (Success)
[02/Jun/2016:12:05:33 -0400] NSMMReplicationPlugin -
agmt="cn=masterAgreement1-ipaserver3.domain.local-pki-ca" (ipaserver3:7389):
Replication bind with SIMPLE auth failed: LDAP error -1 (Can't contact
LDAP server) ((null))
[02/Jun/2016:12:06:01 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 0 (Success)
[02/Jun/2016:12:06:06 -0400] NSMMReplicationPlugin -
agmt="cn=masterAgreement1-ipaserver3.domain.local-pki-ca" (ipaserver3:7389):
Replication bind with SIMPLE auth resumed
[02/Jun/2016:12:06:31 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 107 (Transport
endpoint is not connected)
[02/Jun/2016:12:06:41 -0400] slapi_ldap_bind - Error: could not send
startTLS request: error -1 (Can't contact LDAP server) errno 0 (Success)
NTP seems OK
[God at FirstMasterIPA slapd-PKI-IPA]# date
Thu Jun 2 12:23:00 EDT 2016
[God at ipaserver3 ~]# date
Thu Jun 2 12:23:02 EDT 2016
Sean Hogan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/freeipa-users/attachments/20160620/920bcb0b/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-users/attachments/20160620/920bcb0b/attachment.gif>
More information about the Freeipa-users
mailing list