[Freeipa-users] Kerberos process coredump | authentication fails
Prashant Bapat
prashant at apigee.com
Wed Mar 9 12:36:20 UTC 2016
To follow up on this. I think the issue is resolved.
We have 8 IPA servers. And the primary server on which this error was
occurring had 7 replication agreements! Ended up changing the replication
agreements so that 2 servers had 4 agreements (3 + 1 amongst themselves)
and all others with 2 agreements each.
This seems to have resolved the core-dump of kerberos. No upgrade was done.
Hope this helps someone.
On 29 January 2016 at 15:13, Prashant Bapat <prashant at apigee.com> wrote:
> We will have to run with F21 for now. There are plans for moving to CentOS
> 7.x in the near future. Until then, I'm afraid I will have to live with
> this.
>
> Thanks much Sumit for all your help in identifying this.
>
> Regards.
> --Prashant
>
> On 28 January 2016 at 23:24, Sumit Bose <sbose at redhat.com> wrote:
>
>> On Thu, Jan 28, 2016 at 09:36:55PM +0530, Prashant Bapat wrote:
>> > Sure. Attached the stack trace with debuginfo installed.
>> >
>> > Thanks much!
>>
>> This looks very much like the issue Simo fixed recently, but
>> unfortunately I think it is so recent that it is not available in any
>> release package. Additionally it would be quite some effort for me the
>> generate a F21 test build because as Lukas said F21 is already
>> End-of-life and there is not infrastructure anymore to easily build F21
>> package. If it would be possible to upgrade to a newer version of Fedora
>> I'd be happy to provide a test build with the patch.
>>
>> bye,
>> Sumit
>>
>> >
>> > On 28 January 2016 at 16:53, Sumit Bose <sbose at redhat.com> wrote:
>> >
>> > > On Thu, Jan 28, 2016 at 04:42:20PM +0530, Prashant Bapat wrote:
>> > > > gdb stacktrace attached.
>> > >
>> > > Can you install the debuginfo with
>> > >
>> > > debuginfo-install krb5-server-1.12.2-19.fc21.x86_64
>> > >
>> > > as suggested by gdb and then call 'bt full' again to get more details.
>> > > Additionally the debuginfo of the freeipa package might be missing as
>> > > well.
>> > >
>> > > bye,
>> > > Sumit
>> > > >
>> > > > On 28 January 2016 at 16:27, Prashant Bapat <prashant at apigee.com>
>> wrote:
>> > > >
>> > > > > Thanks Sumit.
>> > > > >
>> > > > > From the logs there is nothing unusual around the time of core
>> dump. I
>> > > > > found this one line odd though.
>> > > > >
>> > > > > *Jan 26 03:15:58 ipa.example.net <http://ipa.example.net>
>> > > > > krb5kdc[4471](Error): worker 4473 exited with status 134*
>> > > > >
>> > > > >
>> > > > > Let me try to get the full BT.
>> > > > >
>> > > > > On 28 January 2016 at 13:54, Sumit Bose <sbose at redhat.com> wrote:
>> > > > >
>> > > > >> On Thu, Jan 28, 2016 at 10:25:53AM +0530, Prashant Bapat wrote:
>> > > > >> > Hi,
>> > > > >> >
>> > > > >> > We have a FreeIPA 4.1.4 setup on F21 servers. There is 1
>> master and
>> > > 7
>> > > > >> > replicas in different regions. Earlier there was only 1
>> replica.
>> > > Since I
>> > > > >> > added new replicas, on the master node, once in a while the
>> kerberos
>> > > > >> > process dumps core and everything stops working -
>> authentication,
>> > > > >> > replication etc. If we restart everything using "ipactl
>> restart"
>> > > things
>> > > > >> are
>> > > > >> > back to normal.
>> > > > >> >
>> > > > >> > Attached is the output from journalctl for kerberos.
>> > > > >> >
>> > > > >> > Has anyone come across this ? Are there any pointers to
>> > > troubleshooting
>> > > > >> > this ?
>> > > > >>
>> > > > >> This might be fixed recently by a patch from Simo
>> > > > >> (2144b1eeb789639b8a3df287b580aeb6196188a8). But to help to better
>> > > > >> identify the issue the content of the kdc logs around the time
>> of the
>> > > > >> crash might be useful. Additionally a full backtrace which you
>> can get
>> > > > >> by calling
>> > > > >>
>> > > > >> coredumpclt gdb 4475
>> > > > >>
>> > > > >> and then
>> > > > >>
>> > > > >> bt full
>> > > > >>
>> > > > >> bye,
>> > > > >> Sumit
>> > > > >>
>> > > > >> >
>> > > > >> > Any help is appreciated.
>> > > > >> >
>> > > > >> > Thanks.
>> > > > >> > --Prashant
>> > > > >>
>> > > > >> > Jan 26 03:15:59 ipa.example.net systemd-coredump[5000]:
>> Process
>> > > 4475
>> > > > >> (krb5kdc) of user 0 dumped core.
>> > > > >> >
>> > > > >> > Stack
>> trace
>> > > of
>> > > > >> thread 4475:
>> > > > >> > #0
>> > > > >> 0x00007f99de8c18d7 raise (libc.so.6)
>> > > > >> > #1
>> > > > >> 0x00007f99de8c353a abort (libc.so.6)
>> > > > >> > #2
>> > > > >> 0x00007f99de8ba47d __assert_fail_base (libc.so.6)
>> > > > >> > #3
>> > > > >> 0x00007f99de8ba532 __assert_fail (libc.so.6)
>> > > > >> > #4
>> > > > >> 0x00007f99d783a78f ldap_get_values_len (libldap_r-2.4.so.2)
>> > > > >> > #5
>> > > > >> 0x00007f99d7c8173e ipadb_ldap_attr_to_int (ipadb.so)
>> > > > >> > #6
>> > > > >> 0x00007f99d7c83f9c ipadb_parse_ldap_entry (ipadb.so)
>> > > > >> > #7
>> > > > >> 0x00007f99d7c849ab ipadb_get_principal (ipadb.so)
>> > > > >> > #8
>> > > > >> 0x00007f99e0433b14 krb5_db_get_principal (libkdb5.so.7)
>> > > > >> > #9
>> > > > >> 0x000055768457c230 process_tgs_req (krb5kdc)
>> > > > >> > #10
>> > > > >> 0x0000557684579fe3 dispatch (krb5kdc)
>> > > > >> > #11
>> > > > >> 0x000055768458d8a0 process_packet (krb5kdc)
>> > > > >> > #12
>> > > > >> 0x00007f99dec4cc78 verto_fire (libverto.so.1)
>> > > > >> > #13
>> > > > >> 0x00007f99d6fb72a3 epoll_event_loop_once (libtevent.so.0)
>> > > > >> > #14
>> > > > >> 0x00007f99d6fb5787 std_event_loop_once (libtevent.so.0)
>> > > > >> > #15
>> > > > >> 0x00007f99d6fb1fed _tevent_loop_once (libtevent.so.0)
>> > > > >> > #16
>> > > > >> 0x00007f99dec4c3f7 verto_run (libverto.so.1)
>> > > > >> > #17
>> > > > >> 0x00005576845795ab main (krb5kdc)
>> > > > >> > #18
>> > > > >> 0x00007f99de8acfe0 __libc_start_main (libc.so.6)
>> > > > >> > #19
>> > > > >> 0x00005576845798f0 _start (krb5kdc)
>> > > > >> >
>> > > > >> > Jan 26 03:15:59 ipa.example.net systemd-coredump[4999]:
>> Process
>> > > 4473
>> > > > >> (krb5kdc) of user 0 dumped core.
>> > > > >> >
>> > > > >> > Stack
>> trace
>> > > of
>> > > > >> thread 4473:
>> > > > >> > #0
>> > > > >> 0x00007f99de8c18d7 raise (libc.so.6)
>> > > > >> > #1
>> > > > >> 0x00007f99de8c353a abort (libc.so.6)
>> > > > >> > #2
>> > > > >> 0x00007f99de8ba47d __assert_fail_base (libc.so.6)
>> > > > >> > #3
>> > > > >> 0x00007f99de8ba532 __assert_fail (libc.so.6)
>> > > > >> > #4
>> > > > >> 0x00007f99d783a78f ldap_get_values_len (libldap_r-2.4.so.2)
>> > > > >> > #5
>> > > > >> 0x00007f99d7c8173e ipadb_ldap_attr_to_int (ipadb.so)
>> > > > >> > #6
>> > > > >> 0x00007f99d7c83f9c ipadb_parse_ldap_entry (ipadb.so)
>> > > > >> > #7
>> > > > >> 0x00007f99d7c849ab ipadb_get_principal (ipadb.so)
>> > > > >> > #8
>> > > > >> 0x00007f99e0433b14 krb5_db_get_principal (libkdb5.so.7)
>> > > > >> > #9
>> > > > >> 0x000055768457c230 process_tgs_req (krb5kdc)
>> > > > >> > #10
>> > > > >> 0x0000557684579fe3 dispatch (krb5kdc)
>> > > > >> > #11
>> > > > >> 0x000055768458d8a0 process_packet (krb5kdc)
>> > > > >> > #12
>> > > > >> 0x00007f99dec4cc78 verto_fire (libverto.so.1)
>> > > > >> > #13
>> > > > >> 0x00007f99d6fb72a3 epoll_event_loop_once (libtevent.so.0)
>> > > > >> > #14
>> > > > >> 0x00007f99d6fb5787 std_event_loop_once (libtevent.so.0)
>> > > > >> > #15
>> > > > >> 0x00007f99d6fb1fed _tevent_loop_once (libtevent.so.0)
>> > > > >> > #16
>> > > > >> 0x00007f99dec4c3f7 verto_run (libverto.so.1)
>> > > > >> > #17
>> > > > >> 0x00005576845795ab main (krb5kdc)
>> > > > >> > #18
>> > > > >> 0x00007f99de8acfe0 __libc_start_main (libc.so.6)
>> > > > >> > #19
>> > > > >> 0x00005576845798f0 _start (krb5kdc)
>> > > > >>
>> > > > >> > --
>> > > > >> > Manage your subscription for the Freeipa-users mailing list:
>> > > > >> > https://www.redhat.com/mailman/listinfo/freeipa-users
>> > > > >> > Go to http://freeipa.org for more info on the project
>> > > > >>
>> > > > >> --
>> > > > >> Manage your subscription for the Freeipa-users mailing list:
>> > > > >> https://www.redhat.com/mailman/listinfo/freeipa-users
>> > > > >> Go to http://freeipa.org for more info on the project
>> > > > >>
>> > > > >
>> > > > >
>> > >
>> > >
>> > >
>> > >
>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/freeipa-users/attachments/20160309/1b083081/attachment.htm>
More information about the Freeipa-users
mailing list