[Freeipa-users] IPA Master Issue - Not starting
Rob Crittenden
rcritten at redhat.com
Wed Aug 20 13:21:14 UTC 2014
Petr Spacek wrote:
> On 20.8.2014 10:02, Peter Grant wrote:
>> Hi Petr,
>>
>> Thanks for your help the other day.
>>
>> Something is bringing down my master instance.
>>
>> i am seeing mismatch on master
>>
>> [root at master init.d]# kvno DNS/master.domain.com at domain.COM
>> DNS/master.domain.com at domain.COM: kvno = 8
>> [root at master init.d]# klist -kt /etc/named.keytab
>> Keytab name: FILE:/etc/named.keytab
>> KVNO Timestamp Principal
>> ---- -----------------
>> --------------------------------------------------------
>> 33 08/20/14 16:41:42 DNS/master.domain.com at domain.COM
>> 33 08/20/14 16:41:42 DNS/master.domain.com at domain.COM
>> 33 08/20/14 16:41:42 DNS/master.domain.com at domain.COM
>> 33 08/20/14 16:41:42 DNS/master.domain.com at domain.COM
>> 34 08/20/14 16:53:29 DNS/master.domain.com at domain.COM
>> 34 08/20/14 16:53:29 DNS/master.domain.com at domain.COM
>> 34 08/20/14 16:53:29 DNS/master.domain.com at domain.COM
>> 34 08/20/14 16:53:29 DNS/master.domain.com at domain.COM
>> 35 08/20/14 16:59:37 DNS/master.domain.com at domain.COM
>> 35 08/20/14 16:59:37 DNS/master.domain.com at domain.COM
>> 35 08/20/14 16:59:37 DNS/master.domain.com at domain.COM
>> 35 08/20/14 16:59:37 DNS/master.domain.com at domain.COM
>> 38 08/20/14 17:02:30 DNS/master.domain.com at domain.COM
>> 38 08/20/14 17:02:30 DNS/master.domain.com at domain.COM
>> 38 08/20/14 17:02:30 DNS/master.domain.com at domain.COM
>> 38 08/20/14 17:02:30 DNS/master.domain.com at domain.COM
>> 41 08/20/14 17:07:45 DNS/master.domain.com at domain.COM
>> 41 08/20/14 17:07:45 DNS/master.domain.com at domain.COM
>> 41 08/20/14 17:07:45 DNS/master.domain.com at domain.COM
>> 41 08/20/14 17:07:45 DNS/master.domain.com at domain.COM
>> 42 08/20/14 17:13:17 DNS/master.domain.com at domain.COM
>> 42 08/20/14 17:13:17 DNS/master.domain.com at domain.COM
>> 42 08/20/14 17:13:17 DNS/master.domain.com at domain.COM
>> 42 08/20/14 17:13:17 DNS/master.domain.com at domain.COM
>> 45 08/20/14 17:20:34 DNS/master.domain.com at domain.COM
>> 45 08/20/14 17:20:34 DNS/master.domain.com at domain.COM
>> 45 08/20/14 17:20:34 DNS/master.domain.com at domain.COM
>> 45 08/20/14 17:20:34 DNS/master.domain.com at domain.COM
>> 46 08/20/14 17:35:00 DNS/master.domain.com at domain.COM
>> 46 08/20/14 17:35:00 DNS/master.domain.com at domain.COM
>> 46 08/20/14 17:35:00 DNS/master.domain.com at domain.COM
>> 46 08/20/14 17:35:00 DNS/master.domain.com at domain.COM
>> 47 08/20/14 17:37:43 DNS/master.domain.com at domain.COM
>> 47 08/20/14 17:37:43 DNS/master.domain.com at domain.COM
>> 47 08/20/14 17:37:43 DNS/master.domain.com at domain.COM
>> 47 08/20/14 17:37:43 DNS/master.domain.com at domain.COM
>> 48 08/20/14 17:41:42 DNS/master.domain.com at domain.COM
>> 48 08/20/14 17:41:42 DNS/master.domain.com at domain.COM
>> 48 08/20/14 17:41:42 DNS/master.domain.com at domain.COM
>> 48 08/20/14 17:41:42 DNS/master.domain.com at domain.COM
>> 49 08/20/14 17:43:43 DNS/master.domain.com at domain.COM
>> 49 08/20/14 17:43:44 DNS/master.domain.com at domain.COM
>> 49 08/20/14 17:43:44 DNS/master.domain.com at domain.COM
>> 49 08/20/14 17:43:44 DNS/master.domain.com at domain.COM
>> [root at master init.d]#
>>
>>
>> also here is output from /var/log/messages whilst trying to ipactl start
>>
>>
>>
>> [root at master init.d]# sudo ipactl start
>> Starting Directory Service
>> Starting dirsrv:
>> domain-COM... [ OK ]
>> PKI-IPA... [ OK ]
>> Starting KDC Service
>> Starting Kerberos 5 KDC: [ OK ]
>> Starting KPASSWD Service
>> Starting Kerberos 5 Admin Server: [ OK ]
>> Starting DNS Service
>> Starting named: 2014-08-20T18:00:22.098747+10:00 master named[20827]:
>> starting BIND 9.8.2rc1-RedHat-9.8.2-0.23.rc1.el6_5.1 -u named
>> 2014-08-20T18:00:22.099552+10:00 master named[20827]: built with
>> '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu'
>> '--target=x86_64-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr'
>> '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin'
>> '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include'
>> '--libdir=/usr/lib64' '--libexecdir=/usr/libexec'
>> '--sharedstatedir=/var/lib' '--mandir=/usr/share/man'
>> '--infodir=/usr/share/info' '--with-libtool' '--localstatedir=/var'
>> '--enable-threads' '--enable-ipv6' '--with-pic' '--disable-static'
>> '--disable-openssl-version-check' '--with-dlz-ldap=yes'
>> '--with-dlz-postgres=yes' '--with-dlz-mysql=yes'
>> '--with-dlz-filesystem=yes' '--with-gssapi=yes' '--disable-isc-spnego'
>> '--with-docbook-xsl=/usr/share/sgml/docbook/xsl-stylesheets'
>> '--enable-fixed-rrset' 'build_alias=x86_64-redhat-linux-gnu'
>> 'host_alias=x86_64-redhat-linux-gnu'
>> 'target_alias=x86_64-redhat-linux-gnu' 'CFLAGS= -O2 -g -pipe -Wall
>> -Wp,-D_FO!
> RTIFY_SOUR
> CE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64
> -mtune=generic' 'CPPFLAGS= -DDIG_SIGCHASE'
>> 2014-08-20T18:00:22.099633+10:00 master named[20827]:
>> ----------------------------------------------------
>> 2014-08-20T18:00:22.099688+10:00 master named[20827]: BIND 9 is
>> maintained by Internet Systems Consortium,
>> 2014-08-20T18:00:22.099750+10:00 master named[20827]: Inc. (ISC), a
>> non-profit 501(c)(3) public-benefit
>> 2014-08-20T18:00:22.099803+10:00 master named[20827]: corporation.
>> Support and training for BIND 9 are
>> 2014-08-20T18:00:22.099864+10:00 master named[20827]: available at
>> https://www.isc.org/support
>> 2014-08-20T18:00:22.099925+10:00 master named[20827]:
>> ----------------------------------------------------
>> 2014-08-20T18:00:22.099998+10:00 master named[20827]: adjusted limit
>> on open files from 62000 to 1048576
>> 2014-08-20T18:00:22.100207+10:00 master named[20827]: found 1 CPU,
>> using 1 worker thread
>> 2014-08-20T18:00:22.100484+10:00 master named[20827]: using up to 4096
>> sockets
>> 2014-08-20T18:00:22.103796+10:00 master named[20827]: loading
>> configuration from '/etc/named.conf'
>> 2014-08-20T18:00:22.104495+10:00 master named[20827]: using default
>> UDP/IPv4 port range: [1024, 65535]
>> 2014-08-20T18:00:22.104728+10:00 master named[20827]: using default
>> UDP/IPv6 port range: [1024, 65535]
>> 2014-08-20T18:00:22.106090+10:00 master named[20827]: listening on
>> IPv6 interfaces, port 53
>> 2014-08-20T18:00:22.108167+10:00 master named[20827]: listening on
>> IPv4 interface lo, 127.0.0.1#53
>> 2014-08-20T18:00:22.108571+10:00 master named[20827]: listening on
>> IPv4 interface eth0, 10.3.11.16#53
>> 2014-08-20T18:00:22.109760+10:00 master named[20827]: generating
>> session key for dynamic DNS
>> 2014-08-20T18:00:22.109997+10:00 master named[20827]: sizing zone task
>> pool based on 5 zones
>> 2014-08-20T18:00:22.112660+10:00 master named[20827]: set up managed
>> keys zone for view _default, file 'dynamic/managed-keys.bind'
>> 2014-08-20T18:00:22.129607+10:00 master named[20827]: Failed to init
>> credentials (Generic preauthentication failure)
>> 2014-08-20T18:00:22.130031+10:00 master named[20827]: loading
>> configuration: failure
>> 2014-08-20T18:00:22.130285+10:00 master named[20827]: exiting (due to
>> fatal error)
>> [FAILED]
>> Failed to start DNS Service
>> Shutting down
>> Stopping Kerberos 5 KDC: [ OK ]
>> Stopping Kerberos 5 Admin Server: 2014-08-20T18:00:23.833115+10:00
>> master ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code
>> may provide more information (Server krbtgt/LOCALDOMAIN at domain.COM not
>> found in Kerberos database)
>
> This seems to be more serious - I suspect that replication between
> replicas doesn't work because replica is not able to authenticate.
>
> The error message is suspicious but I'm not sure that it is not result
> of obfuscation. Please try to apply this article to ns-slapd on your
> broken master:
>
> https://fedorahosted.org/bind-dyndb-ldap/wiki/BIND9/NamedCannotStart#a2.Serverldapsrv01EXAMPLE.COMnotfoundinKerberosdatabase
>
>
> Maybe /etc/hosts is somehow misconfigured.
>
>> [ OK ]
>> Stopping named: [ OK ]
>> Stopping httpd: [FAILED]
>> Stopping pki-ca: [ OK ]
>> Shutting down dirsrv:
>> domain-COM... [ OK ]
>> PKI-IPA... [ OK ]
>> Aborting ipactl
>> [root at master init.d]#
>>
>> however there is still a mismatch when i try to get key tab from
>> secondary using command
>> ipa-getkeytab -s secondary.domain.com -p
>> DNS/master.domain.com at domain.COM -k /etc/named.keytab
>
> Maybe it is caused by broken replication (one KDC have different keys
> than the other KDC). I would start with replication problems and focus
> on named later.
>
I think Petr is right. Consider this. You got a new keytab from the
other master and because this master is down it hasn't been replicated
yet. So this master starts and named tries to use a keytab that the
local master doesn't know about, so things stop.
What I'd do is this:
- Set /etc/resolv.conf to point to working master
# service dirsrv start
# service krb5kdc start
# service httpd start
Wait a bit for replication to take place. You can probably watch the
dirsrv access log for all the gory details. Once things settle down
check on replication status:
# ipa-replica-manage list -v `hostname`
You can check on the kvno per the local KDC using:
# kinit admin
# kvno DNS/`hostname`
The kvno should match the highest one in the keytab. If it doesn't then
that suggests an issue with replication. If it does match, try to start
named. If that comes up ok, I'd manually shut these all off an then use
ipactl to start everything up.
rob
More information about the Freeipa-users
mailing list