[Freeipa-users] krb5kdc will not start (kerberos authentication error)

Ludwig Krispenz lkrispen at redhat.com
Tue Nov 10 14:03:00 UTC 2015 

On 11/10/2015 02:40 PM, Alexander Bokovoy wrote:
> On Tue, 10 Nov 2015, Gronde, Christopher (Contractor) wrote:
>> Where can I verify or change the credentials it is trying to use?  Is 
>> it my LDAP password?
> No, according to your logs, it is your LDAP master trying to replicate
> (push changes) to your LDAP replica:
>>> [09/Nov/2015:15:02:01 -0500] conn=2 fd=64 slot=64 connection from 
>>> <MASTER_IP> to <REPLICA_IP>
>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=0 BIND dn="" method=sasl 
>>> version=3 mech=GSSAPI
err=49 could also be a result if the entry which is mapped from the 
principal is not found in the directory. A bit more info could be gained 
by enabling logging of internal searches.
Set nsslapd-acesslog-level: 260

and then look what internal searches are done during the gssapi 
authentication
>
> If that is true, it would be ldap/<master> Kerberos principal talking to
> ldap/<replica> Kerberos principal. If that fails, it means master and
> replica KDCs have different understanding of both ldap/<master> and
> ldap/<replica> keys which most likely means keys were rotated on master
> and weren't propagated to replica.
>
> How to solve it? One possibility is to set master's hostname as KDC
> address in krb5.conf on replica, forcing LDAP server on replica to use
> master's KDC. I'm absolutely not sure this will actually work but at
> least it allows to see if we are indeed dealing with inconsistent state
> of service principals' keys.
>
>>
>> -----Original Message-----
>> From: Alexander Bokovoy [mailto:abokovoy at redhat.com]
>> Sent: Tuesday, November 10, 2015 8:18 AM
>> To: Gronde, Christopher (Contractor) <Christopher.Gronde at fincen.gov>
>> Cc: Rob Crittenden <rcritten at redhat.com>; freeipa-users at redhat.com
>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos 
>> authentication error)
>>
>> On Tue, 10 Nov 2015, Gronde, Christopher (Contractor) wrote:
>>> When I tried to start the service again I got no response from tail of
>>> the log, but this is a repeating entry I see in the access log
>>>
>>> [09/Nov/2015:15:01:04 -0500] conn=1 fd=64 slot=64 connection from
>>> 127.0.0.1 to 127.0.0.1
>>> [09/Nov/2015:15:01:04 -0500] conn=1 op=-1 fd=64 closed - B1
>>> [09/Nov/2015:15:02:01 -0500] conn=2 fd=64 slot=64 connection from
>>> <MASTER_IP> to <REPLICA_IP>
>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=0 BIND dn="" method=sasl
>>> version=3 mech=GSSAPI
>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=0 RESULT err=14 tag=97
>>> nentries=0 etime=0, SASL bind in progress
>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=1 BIND dn="" method=sasl
>>> version=3 mech=GSSAPI
>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=1 RESULT err=14 tag=97
>>> nentries=0 etime=0, SASL bind in progress
>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=2 BIND dn="" method=sasl
>>> version=3 mech=GSSAPI
>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=2 RESULT err=49 tag=97
>>> nentries=0 etime=0
>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=3 UNBIND
>>> [09/Nov/2015:15:02:01 -0500] conn=2 op=3 fd=64 closed - U1
>>>
>>> Does anyone know what err=14 or err=49 are?
>> err=14 means SASL bind in progress -- i.e. multi-round processing is 
>> ongoing. This is normal for SASL GSSAPI.
>>
>> err=49 is wrong password or username, i.e. credentials were incorrect.
>> It may also mean that LDAP server side was unable to process Kerberos 
>> negotiation due to not having a current Kerberos ticket for own service
>> (LDAP) and trying to request it from the Kerberos KDC but Kerberos 
>> KDC is down.
>>
>>>
>>> -----Original Message-----
>>> From: Rob Crittenden [mailto:rcritten at redhat.com]
>>> Sent: Monday, November 09, 2015 3:26 PM
>>> To: Gronde, Christopher (Contractor) <Christopher.Gronde at fincen.gov>;
>>> Alexander Bokovoy <abokovoy at redhat.com>
>>> Cc: freeipa-users at redhat.com
>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos
>>> authentication error)
>>>
>>> Gronde, Christopher (Contractor) wrote:
>>>> Nothing bad came back and there is definitely data in the tree.
>>>
>>> Ok, I guess I'd try to start the kdc again and then watch the 389-ds 
>>> access log (buffered) to:
>>>
>>> 1. See if it is binding at all
>>> 2. See what the search is and what, if any, results were returned
>>>
>>> This would be in /var/log/dirsrv/slapd-YOUR_REALM/access
>>>
>>> rob
>>>
>>>>
>>>> -----Original Message-----
>>>> From: Rob Crittenden [mailto:rcritten at redhat.com]
>>>> Sent: Monday, November 09, 2015 11:46 AM
>>>> To: Gronde, Christopher (Contractor) <Christopher.Gronde at fincen.gov>;
>>>> Alexander Bokovoy <abokovoy at redhat.com>
>>>> Cc: freeipa-users at redhat.com
>>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos
>>>> authentication error)
>>>>
>>>> Gronde, Christopher (Contractor) wrote:
>>>>> I restarted dirsrv and attempted to start krb5kdc and this is what
>>>>> the error log shows
>>>>>
>>>>> # tail /var/log/dirsrv/slapd-ITMODEV-GOV/errors
>>>>> [09/Nov/2015:11:01:02 -0500] - WARNING: userRoot: entry cache size 
>>>>> 10485760B is less than db size 28016640B; We recommend to increase 
>>>>> the entry cache size nsslapd-cachememsize.
>>>>> [09/Nov/2015:11:01:02 -0500] - slapd started.  Listening on All
>>>>> Interfaces port 389 for LDAP requests
>>>>> [09/Nov/2015:11:06:04 -0500] - slapd shutting down - signaling
>>>>> operation threads
>>>>> [09/Nov/2015:11:06:04 -0500] - slapd shutting down - closing down
>>>>> internal subsystems and plugins
>>>>> [09/Nov/2015:11:06:04 -0500] - Waiting for 4 database threads to
>>>>> stop
>>>>> [09/Nov/2015:11:06:04 -0500] - All database threads now stopped
>>>>> [09/Nov/2015:11:06:04 -0500] - slapd stopped.
>>>>> [09/Nov/2015:11:14:20 -0500] - 389-Directory/1.2.11.15
>>>>> B2015.247.1737 starting up
>>>>> [09/Nov/2015:11:14:20 -0500] - WARNING: userRoot: entry cache size 
>>>>> 10485760B is less than db size 28016640B; We recommend to increase 
>>>>> the entry cache size nsslapd-cachememsize.
>>>>> [09/Nov/2015:11:14:20 -0500] - slapd started.  Listening on All
>>>>> Interfaces port 389 for LDAP requests
>>>>
>>>> Ok, that's good.
>>>>
>>>> I'd do something like this to see what is in the db (substitute 
>>>> example.com with your domain):
>>>>
>>>> $ ldapsearch -x -D 'cn=Directory Manager' -W -s one -b
>>>> cn=kerberos,dc=example,dc=com
>>>>
>>>> (don't post the output as it would include the kerberos master key).
>>>>
>>>> If that returns nothing that's bad.
>>>>
>>>> If it succeeds I'd broaden the search base a bit to see what data you
>>>> do
>>>> have:
>>>>
>>>> $ ldapsearch -x -D 'cn=Directory Manager' -W -b
>>>> cn=groups,cn=accounts,dc=example,dc=com
>>>>
>>>> I picked groups because usually groups << users in numbers. This is 
>>>> just to see if you have data in the tree.
>>>>
>>>> Let us know if either or both turns up nothing.
>>>>
>>>> rob
>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: Alexander Bokovoy [mailto:abokovoy at redhat.com]
>>>>> Sent: Monday, November 09, 2015 10:51 AM
>>>>> To: Gronde, Christopher (Contractor) <Christopher.Gronde at fincen.gov>
>>>>> Cc: freeipa-users at redhat.com
>>>>> Subject: Re: [Freeipa-users] krb5kdc will not start (kerberos
>>>>> authentication error)
>>>>>
>>>>> On Mon, 09 Nov 2015, Gronde, Christopher (Contractor) wrote:
>>>>>> Hello all!
>>>>>>
>>>>>> On my replica IPA server after fixing a cert issue that had been 
>>>>>> going on for sometime, I have all my certs figured out but the 
>>>>>> krb5kdc service will not start.
>>>>>>
>>>>>> # service krb5kdc start
>>>>>> Starting Kerberos 5 KDC: krb5kdc: cannot initialize realm 
>>>>>> ITMODEV.GOV - see log file for details                  [FAILED]
>>>>>>
>>>>>> # cat /var/log/krb5kdc.log
>>>>>> krb5kdc: Server error - while fetching master key K/M for realm
>>>>>> ITMODEV.GOV
>>>>>> krb5kdc: Server error - while fetching master key K/M for realm
>>>>>> ITMODEV.GOV
>>>>>> krb5kdc: Server error - while fetching master key K/M for realm
>>>>>> ITMODEV.GOV
>>>>>>
>>>>>> I found this article online:
>>>>>> http://research.imb.uq.edu.au/~l.rathbone/ldap/kerberos.shtml
>>>>>>
>>>>>> Which stated it might be because The slave KDC does not have a
>>>>>> stash file (.k5.EXAMPLE.COM). You need to create one. Tried the
>>>>>> command
>>>>>> listed:
>>>>>>
>>>>>> # kdb5_util stash
>>>>>> kdb5_util: Server error while retrieving master entry
>>>>>>
>>>>>> No further information found on the proceeding error above for 
>>>>>> the kdb5_util command.
>>>>>>
>>>>>> Any thoughts?
>>>>> First: don't use instructions which are not related to IPA, please.
>>>>>
>>>>> FreeIPA has its own LDAP driver for KDC and instructions for 
>>>>> anything else do not apply here at all.
>>>>>
>>>>> If you see 'Server error - while fetching master key ..' it means 
>>>>> KDC LDAP driver was unable to contact LDAP server. Does LDAP 
>>>>> server work on the replica? What is in its error log 
>>>>> (/var/log/dirsrv/slapd-ITMODEV-GOV/errors)?
>>>>>
>>>>> -- 
>>>>> / Alexander Bokovoy
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>> -- 
>> / Alexander Bokovoy
>>
>

More information about the Freeipa-users mailing list