[Freeipa-users] krb5kdc will not start (kerberos authentication error)

Ludwig Krispenz lkrispen at redhat.com
Tue Nov 10 14:47:38 UTC 2015 

On 11/10/2015 03:32 PM, Gronde, Christopher (Contractor) wrote:
> How do I change that log setting?  Is that done in LDAP?  Using ldapmodify?
yes,
ldapmodify ...
dn: cn=config
changetype: modify
replace: nsslapd-acesslog-level
nsslapd-acesslog-level: 260
>
> -----Original Message-----
> From: freeipa-users-bounces at redhat.com [mailto:freeipa-users-bounces at redhat.com] On Behalf Of Ludwig Krispenz
> Sent: Tuesday, November 10, 2015 9:03 AM
> To: freeipa-users at redhat.com
> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos authentication error)
>
>
> On 11/10/2015 02:40 PM, Alexander Bokovoy wrote:
>> On Tue, 10 Nov 2015, Gronde, Christopher (Contractor) wrote:
>>> Where can I verify or change the credentials it is trying to use?  Is
>>> it my LDAP password?
>> No, according to your logs, it is your LDAP master trying to replicate
>> (push changes) to your LDAP replica:
>>>> [09/Nov/2015:15:02:01 -0500] conn=2 fd=64 slot=64 connection from
>>>> <MASTER_IP> to <REPLICA_IP>
>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=0 BIND dn="" method=sasl
>>>> version=3 mech=GSSAPI
> err=49 could also be a result if the entry which is mapped from the principal is not found in the directory. A bit more info could be gained by enabling logging of internal searches.
> Set nsslapd-acesslog-level: 260
>
> and then look what internal searches are done during the gssapi authentication
>> If that is true, it would be ldap/<master> Kerberos principal talking
>> to ldap/<replica> Kerberos principal. If that fails, it means master
>> and replica KDCs have different understanding of both ldap/<master>
>> and ldap/<replica> keys which most likely means keys were rotated on
>> master and weren't propagated to replica.
>>
>> How to solve it? One possibility is to set master's hostname as KDC
>> address in krb5.conf on replica, forcing LDAP server on replica to use
>> master's KDC. I'm absolutely not sure this will actually work but at
>> least it allows to see if we are indeed dealing with inconsistent
>> state of service principals' keys.
>>
>>> -----Original Message-----
>>> From: Alexander Bokovoy [mailto:abokovoy at redhat.com]
>>> Sent: Tuesday, November 10, 2015 8:18 AM
>>> To: Gronde, Christopher (Contractor) <Christopher.Gronde at fincen.gov>
>>> Cc: Rob Crittenden <rcritten at redhat.com>; freeipa-users at redhat.com
>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos
>>> authentication error)
>>>
>>> On Tue, 10 Nov 2015, Gronde, Christopher (Contractor) wrote:
>>>> When I tried to start the service again I got no response from tail
>>>> of the log, but this is a repeating entry I see in the access log
>>>>
>>>> [09/Nov/2015:15:01:04 -0500] conn=1 fd=64 slot=64 connection from
>>>> 127.0.0.1 to 127.0.0.1
>>>> [09/Nov/2015:15:01:04 -0500] conn=1 op=-1 fd=64 closed - B1
>>>> [09/Nov/2015:15:02:01 -0500] conn=2 fd=64 slot=64 connection from
>>>> <MASTER_IP> to <REPLICA_IP>
>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=0 BIND dn="" method=sasl
>>>> version=3 mech=GSSAPI
>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=0 RESULT err=14 tag=97
>>>> nentries=0 etime=0, SASL bind in progress
>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=1 BIND dn="" method=sasl
>>>> version=3 mech=GSSAPI
>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=1 RESULT err=14 tag=97
>>>> nentries=0 etime=0, SASL bind in progress
>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=2 BIND dn="" method=sasl
>>>> version=3 mech=GSSAPI
>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=2 RESULT err=49 tag=97
>>>> nentries=0 etime=0
>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=3 UNBIND
>>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=3 fd=64 closed - U1
>>>>
>>>> Does anyone know what err=14 or err=49 are?
>>> err=14 means SASL bind in progress -- i.e. multi-round processing is
>>> ongoing. This is normal for SASL GSSAPI.
>>>
>>> err=49 is wrong password or username, i.e. credentials were incorrect.
>>> It may also mean that LDAP server side was unable to process Kerberos
>>> negotiation due to not having a current Kerberos ticket for own
>>> service
>>> (LDAP) and trying to request it from the Kerberos KDC but Kerberos
>>> KDC is down.
>>>
>>>> -----Original Message-----
>>>> From: Rob Crittenden [mailto:rcritten at redhat.com]
>>>> Sent: Monday, November 09, 2015 3:26 PM
>>>> To: Gronde, Christopher (Contractor)
>>>> <Christopher.Gronde at fincen.gov>; Alexander Bokovoy
>>>> <abokovoy at redhat.com>
>>>> Cc: freeipa-users at redhat.com
>>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos
>>>> authentication error)
>>>>
>>>> Gronde, Christopher (Contractor) wrote:
>>>>> Nothing bad came back and there is definitely data in the tree.
>>>> Ok, I guess I'd try to start the kdc again and then watch the 389-ds
>>>> access log (buffered) to:
>>>>
>>>> 1. See if it is binding at all
>>>> 2. See what the search is and what, if any, results were returned
>>>>
>>>> This would be in /var/log/dirsrv/slapd-YOUR_REALM/access
>>>>
>>>> rob
>>>>
>>>>> -----Original Message-----
>>>>> From: Rob Crittenden [mailto:rcritten at redhat.com]
>>>>> Sent: Monday, November 09, 2015 11:46 AM
>>>>> To: Gronde, Christopher (Contractor)
>>>>> <Christopher.Gronde at fincen.gov>; Alexander Bokovoy
>>>>> <abokovoy at redhat.com>
>>>>> Cc: freeipa-users at redhat.com
>>>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos
>>>>> authentication error)
>>>>>
>>>>> Gronde, Christopher (Contractor) wrote:
>>>>>> I restarted dirsrv and attempted to start krb5kdc and this is what
>>>>>> the error log shows
>>>>>>
>>>>>> # tail /var/log/dirsrv/slapd-ITMODEV-GOV/errors
>>>>>> [09/Nov/2015:11:01:02 -0500] - WARNING: userRoot: entry cache size
>>>>>> 10485760B is less than db size 28016640B; We recommend to increase
>>>>>> the entry cache size nsslapd-cachememsize.
>>>>>> [09/Nov/2015:11:01:02 -0500] - slapd started.  Listening on All
>>>>>> Interfaces port 389 for LDAP requests
>>>>>> [09/Nov/2015:11:06:04 -0500] - slapd shutting down - signaling
>>>>>> operation threads
>>>>>> [09/Nov/2015:11:06:04 -0500] - slapd shutting down - closing down
>>>>>> internal subsystems and plugins
>>>>>> [09/Nov/2015:11:06:04 -0500] - Waiting for 4 database threads to
>>>>>> stop
>>>>>> [09/Nov/2015:11:06:04 -0500] - All database threads now stopped
>>>>>> [09/Nov/2015:11:06:04 -0500] - slapd stopped.
>>>>>> [09/Nov/2015:11:14:20 -0500] - 389-Directory/1.2.11.15
>>>>>> B2015.247.1737 starting up
>>>>>> [09/Nov/2015:11:14:20 -0500] - WARNING: userRoot: entry cache size
>>>>>> 10485760B is less than db size 28016640B; We recommend to increase
>>>>>> the entry cache size nsslapd-cachememsize.
>>>>>> [09/Nov/2015:11:14:20 -0500] - slapd started.  Listening on All
>>>>>> Interfaces port 389 for LDAP requests
>>>>> Ok, that's good.
>>>>>
>>>>> I'd do something like this to see what is in the db (substitute
>>>>> example.com with your domain):
>>>>>
>>>>> $ ldapsearch -x -D 'cn=Directory Manager' -W -s one -b
>>>>> cn=kerberos,dc=example,dc=com
>>>>>
>>>>> (don't post the output as it would include the kerberos master key).
>>>>>
>>>>> If that returns nothing that's bad.
>>>>>
>>>>> If it succeeds I'd broaden the search base a bit to see what data
>>>>> you do
>>>>> have:
>>>>>
>>>>> $ ldapsearch -x -D 'cn=Directory Manager' -W -b
>>>>> cn=groups,cn=accounts,dc=example,dc=com
>>>>>
>>>>> I picked groups because usually groups << users in numbers. This is
>>>>> just to see if you have data in the tree.
>>>>>
>>>>> Let us know if either or both turns up nothing.
>>>>>
>>>>> rob
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Alexander Bokovoy [mailto:abokovoy at redhat.com]
>>>>>> Sent: Monday, November 09, 2015 10:51 AM
>>>>>> To: Gronde, Christopher (Contractor)
>>>>>> <Christopher.Gronde at fincen.gov>
>>>>>> Cc: freeipa-users at redhat.com
>>>>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos
>>>>>> authentication error)
>>>>>>
>>>>>> On Mon, 09 Nov 2015, Gronde, Christopher (Contractor) wrote:
>>>>>>> Hello all!
>>>>>>>
>>>>>>> On my replica IPA server after fixing a cert issue that had been
>>>>>>> going on for sometime, I have all my certs figured out but the
>>>>>>> krb5kdc service will not start.
>>>>>>>
>>>>>>> # service krb5kdc start
>>>>>>> Starting Kerberos 5 KDC: krb5kdc: cannot initialize realm
>>>>>>> ITMODEV.GOV - see log file for details                  [FAILED]
>>>>>>>
>>>>>>> # cat /var/log/krb5kdc.log
>>>>>>> krb5kdc: Server error - while fetching master key K/M for realm
>>>>>>> ITMODEV.GOV
>>>>>>> krb5kdc: Server error - while fetching master key K/M for realm
>>>>>>> ITMODEV.GOV
>>>>>>> krb5kdc: Server error - while fetching master key K/M for realm
>>>>>>> ITMODEV.GOV
>>>>>>>
>>>>>>> I found this article online:
>>>>>>> http://research.imb.uq.edu.au/~l.rathbone/ldap/kerberos.shtml
>>>>>>>
>>>>>>> Which stated it might be because The slave KDC does not have a
>>>>>>> stash file (.k5.EXAMPLE.COM). You need to create one. Tried the
>>>>>>> command
>>>>>>> listed:
>>>>>>>
>>>>>>> # kdb5_util stash
>>>>>>> kdb5_util: Server error while retrieving master entry
>>>>>>>
>>>>>>> No further information found on the proceeding error above for
>>>>>>> the kdb5_util command.
>>>>>>>
>>>>>>> Any thoughts?
>>>>>> First: don't use instructions which are not related to IPA, please.
>>>>>>
>>>>>> FreeIPA has its own LDAP driver for KDC and instructions for
>>>>>> anything else do not apply here at all.
>>>>>>
>>>>>> If you see 'Server error - while fetching master key ..' it means
>>>>>> KDC LDAP driver was unable to contact LDAP server. Does LDAP
>>>>>> server work on the replica? What is in its error log
>>>>>> (/var/log/dirsrv/slapd-ITMODEV-GOV/errors)?
>>>>>>
>>>>>> --
>>>>>> / Alexander Bokovoy
>>>>>>
>>>>>>
>>>>>
>>>>
>>> --
>>> / Alexander Bokovoy
>>>
> --
> Manage your subscription for the Freeipa-users mailing list:
> https://www.redhat.com/mailman/listinfo/freeipa-users
> Go to http://freeipa.org for more info on the project
>

More information about the Freeipa-users mailing list