[Freeipa-users] Kerberos hanging approx. once a day
Sumit Bose
sbose at redhat.com
Wed Jul 22 09:21:20 UTC 2015
On Wed, Jul 22, 2015 at 11:06:53AM +0200, Torsten Harenberg wrote:
> Dear community,
>
> we just moved our infrastructure (about 200 node cluster plus about 30
> workstations) from NIS to FreeIPA (version 4.1.4 on FC 21).
>
> We have two IPA servers (called "ipa" and "ipa2" both paravirtualized on
> Xen4).
>
> Approx once a day, the Kerberos service on the primary server suddenly
> stops working and I am unable to re-start the service. Only a "full"
> reboot helps and during that, the Kerberos shutdown takes about 2
> minutes (unsure if it really finishes or if it's the final timeout of
> the shutdown script).
>
> Trying to collect as many log messages as possible:
>
>
> Jul 22 10:52:06 ipa.pleiades.uni-wuppertal.de krb5kdc[1114](info):
> AS_REQ (4 etypes {18 17 16 23}) 132.195.124.213: LOOKING_UP_CLIENT:
> host/proton.pleiades.uni-wuppertal.de at PLEIADES.UNI-WUPPERTAL.DE for
> krbtgt/PLEIADES.UNI-WUPPERTAL.DE at PLEIADES.UNI-WUPPERTAL.DE, Server error
> Jul 22 10:52:11 ipa.pleiades.uni-wuppertal.de krb5kdc[1114](info):
> AS_REQ (4 etypes {18 17 16 23}) 132.195.125.171: LOOKING_UP_CLIENT:
> host/wn161.pleiades.uni-wuppertal.de at PLEIADES.UNI-WUPPERTAL.DE for
> krbtgt/PLEIADES.UNI-WUPPERTAL.DE at PLEIADES.UNI-WUPPERTAL.DE, Server error
Looks like there are issues getting the needed data from the local LDAP
server. The message below about the master key points into the same
direction. Can you check the 389ds logs?
bye,
Sumit
>
>
> [root at ipa ~]# systemctl status krb5kdc.service
> ● krb5kdc.service - Kerberos 5 KDC
> Loaded: loaded (/usr/lib/systemd/system/krb5kdc.service; disabled)
> Active: failed (Result: exit-code) since Mi 2015-07-22 10:54:22 CEST;
> 10s ago
> Process: 11910 ExecStart=/usr/sbin/krb5kdc -P /var/run/krb5kdc.pid
> $KRB5KDC_ARGS (code=exited, status=1/FAILURE)
> Main PID: 1114 (code=exited, status=0/SUCCESS)
>
> Jul 22 10:54:22 ipa.pleiades.uni-wuppertal.de krb5kdc[11910]: krb5kdc:
> cannot initialize realm PLEIADES.UNI-WUPPERTAL.DE - see log file for details
> Jul 22 10:54:22 ipa.pleiades.uni-wuppertal.de systemd[1]:
> krb5kdc.service: control process exited, code=exited status=1
> Jul 22 10:54:22 ipa.pleiades.uni-wuppertal.de systemd[1]: Failed to
> start Kerberos 5 KDC.
> Jul 22 10:54:22 ipa.pleiades.uni-wuppertal.de systemd[1]: Unit
> krb5kdc.service entered failed state.
> Jul 22 10:54:22 ipa.pleiades.uni-wuppertal.de systemd[1]:
> krb5kdc.service failed.
> [root at ipa ~]# tail -f /var/log/krb5kdc.log
> Jul 22 10:52:11 ipa.pleiades.uni-wuppertal.de krb5kdc[1114](info):
> closing down fd 13
> Jul 22 10:52:11 ipa.pleiades.uni-wuppertal.de krb5kdc[1114](info):
> DISPATCH: repeated (retransmitted?) request from 132.195.124.213,
> resending previous response
> Jul 22 10:52:11 ipa.pleiades.uni-wuppertal.de krb5kdc[1114](info):
> DISPATCH: repeated (retransmitted?) request from 132.195.125.171,
> resending previous response
> Jul 22 10:54:12 ipa.pleiades.uni-wuppertal.de krb5kdc[1114](debug): Got
> signal to request exit
> Jul 22 10:54:12 ipa.pleiades.uni-wuppertal.de krb5kdc[1114](info):
> closing down fd 10
> Jul 22 10:54:12 ipa.pleiades.uni-wuppertal.de krb5kdc[1114](info):
> closing down fd 11
> Jul 22 10:54:12 ipa.pleiades.uni-wuppertal.de krb5kdc[1114](info):
> closing down fd 9
> Jul 22 10:54:12 ipa.pleiades.uni-wuppertal.de krb5kdc[1114](info):
> closing down fd 8
> Jul 22 10:54:12 ipa.pleiades.uni-wuppertal.de krb5kdc[1114](info):
> shutting down
> krb5kdc: Server error - while fetching master key K/M for realm
> PLEIADES.UNI-WUPPERTAL.DE
>
>
> [root at ipa ~]# journalctl -xe
> Jul 22 10:53:15 ipa.pleiades.uni-wuppertal.de ntpd[11903]: Listen and
> drop on 1 v6wildcard :: UDP 123
> Jul 22 10:53:15 ipa.pleiades.uni-wuppertal.de ntpd[11903]: Listen
> normally on 2 lo 127.0.0.1 UDP 123
> Jul 22 10:53:15 ipa.pleiades.uni-wuppertal.de ntpd[11903]: Listen
> normally on 3 eth0 132.195.124.12 UDP 123
> Jul 22 10:53:15 ipa.pleiades.uni-wuppertal.de ntpd[11903]: Listen
> normally on 4 lo ::1 UDP 123
> Jul 22 10:53:15 ipa.pleiades.uni-wuppertal.de ntpd[11903]: Listen
> normally on 5 eth0 fe80::216:3eff:fe14:c27a UDP 123
> Jul 22 10:53:15 ipa.pleiades.uni-wuppertal.de ntpd[11903]: Listening on
> routing socket on fd #22 for interface updates
> Jul 22 10:53:15 ipa.pleiades.uni-wuppertal.de systemd[1]: Started
> Network Time Service.
> -- Subject: Unit ntpd.service has finished start-up
> -- Defined-By: systemd
> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
> --
> -- Unit ntpd.service has finished starting up.
> --
> -- The start-up result is done.
> Jul 22 10:53:15 ipa.pleiades.uni-wuppertal.de ntpd[11903]: 0.0.0.0 c016
> 06 restart
> Jul 22 10:53:15 ipa.pleiades.uni-wuppertal.de ntpd[11903]: 0.0.0.0 c012
> 02 freq_set ntpd -23.557 PPM
> Jul 22 10:53:16 ipa.pleiades.uni-wuppertal.de ntpd[11903]: 0.0.0.0 c615
> 05 clock_sync
> Jul 22 10:54:12 ipa.pleiades.uni-wuppertal.de systemd[1]: Stopping
> Kerberos 5 KDC...
> -- Subject: Unit krb5kdc.service has begun shutting down
> -- Defined-By: systemd
> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
> --
> -- Unit krb5kdc.service has begun shutting down.
> Jul 22 10:54:12 ipa.pleiades.uni-wuppertal.de systemd[1]: Starting
> Kerberos 5 KDC...
> -- Subject: Unit krb5kdc.service has begun with start-up
> -- Defined-By: systemd
> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
> --
> -- Unit krb5kdc.service has begun starting up.
> Jul 22 10:54:22 ipa.pleiades.uni-wuppertal.de krb5kdc[11910]: krb5kdc:
> cannot initialize realm PLEIADES.UNI-WUPPERTAL.DE - see log file for details
> Jul 22 10:54:22 ipa.pleiades.uni-wuppertal.de systemd[1]:
> krb5kdc.service: control process exited, code=exited status=1
> Jul 22 10:54:22 ipa.pleiades.uni-wuppertal.de systemd[1]: Failed to
> start Kerberos 5 KDC.
> -- Subject: Unit krb5kdc.service has failed
> -- Defined-By: systemd
> -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
> --
> -- Unit krb5kdc.service has failed.
> --
> -- The result is failed.
> Jul 22 10:54:22 ipa.pleiades.uni-wuppertal.de systemd[1]: Unit
> krb5kdc.service entered failed state.
> Jul 22 10:54:22 ipa.pleiades.uni-wuppertal.de systemd[1]:
> krb5kdc.service failed.
> [root at ipa ~]#
>
>
> [root at ipa ~]# rpm -qi freeipa-server
> Name : freeipa-server
> Version : 4.1.4
> Release : 1.fc21
> Architecture: x86_64
> Install Date: Di 28 Apr 2015 14:30:33 CEST
> Group : System Environment/Base
> Size : 4521059
> License : GPLv3+
> Signature : RSA/SHA256, Do 26 Mär 2015 23:58:02 CET, Key ID
> 89ad4e8795a43f54
> Source RPM : freeipa-4.1.4-1.fc21.src.rpm
> Build Date : Do 26 Mär 2015 16:16:19 CET
> Build Host : buildhw-07.phx2.fedoraproject.org
> Relocations : (not relocatable)
> Packager : Fedora Project
> Vendor : Fedora Project
> URL : http://www.freeipa.org/
> Summary : The IPA authentication server
> Description :
> IPA is an integrated solution to provide centrally managed Identity
> (machine,
> user, virtual machines, groups, authentication credentials), Policy
> (configuration settings, access control information) and Audit (events,
> logs, analysis thereof). If you are installing an IPA server you need
> to install this package (in other words, most people should NOT install
> this package).
> [root at ipa ~]#
>
> We already enlarged the capacity of the primary server (now two
> exclusive CPU cores and 8 GB RAM).
>
> Any idea is appreciated, we are pretty new to IPA.
>
> Kind regards,
>
> Torsten
>
>
> --
> <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
> <> <>
> <> Dr. Torsten Harenberg harenberg at physik.uni-wuppertal.de <>
> <> Bergische Universitaet <>
> <> FB C - Physik Tel.: +49 (0)202 439-3521 <>
> <> Gaussstr. 20 Fax : +49 (0)202 439-2811 <>
> <> 42097 Wuppertal <>
> <> <>
> <><><><><><><>< Of course it runs NetBSD http://www.netbsd.org ><>
>
> --
> Manage your subscription for the Freeipa-users mailing list:
> https://www.redhat.com/mailman/listinfo/freeipa-users
> Go to http://freeipa.org for more info on the project
More information about the Freeipa-users
mailing list