<div dir="ltr"><div class="gmail_default" style="font-family:'trebuchet ms',sans-serif">We will have to run with F21 for now. There are plans for moving to CentOS 7.x in the near future. Until then, I'm afraid I will have to live with this. </div><div class="gmail_default" style="font-family:'trebuchet ms',sans-serif"> </div><div class="gmail_default" style="font-family:'trebuchet ms',sans-serif">Thanks much Sumit for all your help in identifying this.</div><div class="gmail_default" style="font-family:'trebuchet ms',sans-serif"> </div><div class="gmail_default" style="font-family:'trebuchet ms',sans-serif">Regards.</div><div class="gmail_default" style="font-family:'trebuchet ms',sans-serif">--Prashant</div></div><div class="gmail_extra"> <div class="gmail_quote">On 28 January 2016 at 23:24, Sumit Bose <<a href="mailto:sbose@redhat.com" target="_blank">sbose@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Thu, Jan 28, 2016 at 09:36:55PM +0530, Prashant Bapat wrote: > Sure. Attached the stack trace with debuginfo installed. > > Thanks much! This looks very much like the issue Simo fixed recently, but unfortunately I think it is so recent that it is not available in any release package. Additionally it would be quite some effort for me the generate a F21 test build because as Lukas said F21 is already End-of-life and there is not infrastructure anymore to easily build F21 package. If it would be possible to upgrade to a newer version of Fedora I'd be happy to provide a test build with the patch. <div class="HOEnZb"><div class="h5"> bye, Sumit > > On 28 January 2016 at 16:53, Sumit Bose <<a href="mailto:sbose@redhat.com">sbose@redhat.com</a>> wrote: > > > On Thu, Jan 28, 2016 at 04:42:20PM +0530, Prashant Bapat wrote: > > > gdb stacktrace attached. > > > > Can you install the debuginfo with > > > > debuginfo-install krb5-server-1.12.2-19.fc21.x86_64 > > > > as suggested by gdb and then call 'bt full' again to get more details. > > Additionally the debuginfo of the freeipa package might be missing as > > well. > > > > bye, > > Sumit > > > > > > On 28 January 2016 at 16:27, Prashant Bapat <<a href="mailto:prashant@apigee.com">prashant@apigee.com</a>> wrote: > > > > > > > Thanks Sumit. > > > > > > > > From the logs there is nothing unusual around the time of core dump. I > > > > found this one line odd though. > > > > > > > > *Jan 26 03:15:58 <a href="http://ipa.example.net" rel="noreferrer" target="_blank">ipa.example.net</a> <<a href="http://ipa.example.net" rel="noreferrer" target="_blank">http://ipa.example.net</a>> > > > > krb5kdc[4471](Error): worker 4473 exited with status 134* > > > > > > > > > > > > Let me try to get the full BT. > > > > > > > > On 28 January 2016 at 13:54, Sumit Bose <<a href="mailto:sbose@redhat.com">sbose@redhat.com</a>> wrote: > > > > > > > >> On Thu, Jan 28, 2016 at 10:25:53AM +0530, Prashant Bapat wrote: > > > >> > Hi, > > > >> > > > > >> > We have a FreeIPA 4.1.4 setup on F21 servers. There is 1 master and > > 7 > > > >> > replicas in different regions. Earlier there was only 1 replica. > > Since I > > > >> > added new replicas, on the master node, once in a while the kerberos > > > >> > process dumps core and everything stops working - authentication, > > > >> > replication etc. If we restart everything using "ipactl restart" > > things > > > >> are > > > >> > back to normal. > > > >> > > > > >> > Attached is the output from journalctl for kerberos. > > > >> > > > > >> > Has anyone come across this ? Are there any pointers to > > troubleshooting > > > >> > this ? > > > >> > > > >> This might be fixed recently by a patch from Simo > > > >> (2144b1eeb789639b8a3df287b580aeb6196188a8). But to help to better > > > >> identify the issue the content of the kdc logs around the time of the > > > >> crash might be useful. Additionally a full backtrace which you can get > > > >> by calling > > > >> > > > >> coredumpclt gdb 4475 > > > >> > > > >> and then > > > >> > > > >> bt full > > > >> > > > >> bye, > > > >> Sumit > > > >> > > > >> > > > > >> > Any help is appreciated. > > > >> > > > > >> > Thanks. > > > >> > --Prashant > > > >> > > > >> > Jan 26 03:15:59 <a href="http://ipa.example.net" rel="noreferrer" target="_blank">ipa.example.net</a> systemd-coredump[5000]: Process > > 4475 > > > >> (krb5kdc) of user 0 dumped core. > > > >> > > > > >> > Stack trace > > of > > > >> thread 4475: > > > >> > #0 > > > >> 0x00007f99de8c18d7 raise (libc.so.6) > > > >> > #1 > > > >> 0x00007f99de8c353a abort (libc.so.6) > > > >> > #2 > > > >> 0x00007f99de8ba47d __assert_fail_base (libc.so.6) > > > >> > #3 > > > >> 0x00007f99de8ba532 __assert_fail (libc.so.6) > > > >> > #4 > > > >> 0x00007f99d783a78f ldap_get_values_len (libldap_r-2.4.so.2) > > > >> > #5 > > > >> 0x00007f99d7c8173e ipadb_ldap_attr_to_int (ipadb.so) > > > >> > #6 > > > >> 0x00007f99d7c83f9c ipadb_parse_ldap_entry (ipadb.so) > > > >> > #7 > > > >> 0x00007f99d7c849ab ipadb_get_principal (ipadb.so) > > > >> > #8 > > > >> 0x00007f99e0433b14 krb5_db_get_principal (libkdb5.so.7) > > > >> > #9 > > > >> 0x000055768457c230 process_tgs_req (krb5kdc) > > > >> > #10 > > > >> 0x0000557684579fe3 dispatch (krb5kdc) > > > >> > #11 > > > >> 0x000055768458d8a0 process_packet (krb5kdc) > > > >> > #12 > > > >> 0x00007f99dec4cc78 verto_fire (libverto.so.1) > > > >> > #13 > > > >> 0x00007f99d6fb72a3 epoll_event_loop_once (libtevent.so.0) > > > >> > #14 > > > >> 0x00007f99d6fb5787 std_event_loop_once (libtevent.so.0) > > > >> > #15 > > > >> 0x00007f99d6fb1fed _tevent_loop_once (libtevent.so.0) > > > >> > #16 > > > >> 0x00007f99dec4c3f7 verto_run (libverto.so.1) > > > >> > #17 > > > >> 0x00005576845795ab main (krb5kdc) > > > >> > #18 > > > >> 0x00007f99de8acfe0 __libc_start_main (libc.so.6) > > > >> > #19 > > > >> 0x00005576845798f0 _start (krb5kdc) > > > >> > > > > >> > Jan 26 03:15:59 <a href="http://ipa.example.net" rel="noreferrer" target="_blank">ipa.example.net</a> systemd-coredump[4999]: Process > > 4473 > > > >> (krb5kdc) of user 0 dumped core. > > > >> > > > > >> > Stack trace > > of > > > >> thread 4473: > > > >> > #0 > > > >> 0x00007f99de8c18d7 raise (libc.so.6) > > > >> > #1 > > > >> 0x00007f99de8c353a abort (libc.so.6) > > > >> > #2 > > > >> 0x00007f99de8ba47d __assert_fail_base (libc.so.6) > > > >> > #3 > > > >> 0x00007f99de8ba532 __assert_fail (libc.so.6) > > > >> > #4 > > > >> 0x00007f99d783a78f ldap_get_values_len (libldap_r-2.4.so.2) > > > >> > #5 > > > >> 0x00007f99d7c8173e ipadb_ldap_attr_to_int (ipadb.so) > > > >> > #6 > > > >> 0x00007f99d7c83f9c ipadb_parse_ldap_entry (ipadb.so) > > > >> > #7 > > > >> 0x00007f99d7c849ab ipadb_get_principal (ipadb.so) > > > >> > #8 > > > >> 0x00007f99e0433b14 krb5_db_get_principal (libkdb5.so.7) > > > >> > #9 > > > >> 0x000055768457c230 process_tgs_req (krb5kdc) > > > >> > #10 > > > >> 0x0000557684579fe3 dispatch (krb5kdc) > > > >> > #11 > > > >> 0x000055768458d8a0 process_packet (krb5kdc) > > > >> > #12 > > > >> 0x00007f99dec4cc78 verto_fire (libverto.so.1) > > > >> > #13 > > > >> 0x00007f99d6fb72a3 epoll_event_loop_once (libtevent.so.0) > > > >> > #14 > > > >> 0x00007f99d6fb5787 std_event_loop_once (libtevent.so.0) > > > >> > #15 > > > >> 0x00007f99d6fb1fed _tevent_loop_once (libtevent.so.0) > > > >> > #16 > > > >> 0x00007f99dec4c3f7 verto_run (libverto.so.1) > > > >> > #17 > > > >> 0x00005576845795ab main (krb5kdc) > > > >> > #18 > > > >> 0x00007f99de8acfe0 __libc_start_main (libc.so.6) > > > >> > #19 > > > >> 0x00005576845798f0 _start (krb5kdc) > > > >> > > > >> > -- > > > >> > Manage your subscription for the Freeipa-users mailing list: > > > >> > <a href="https://www.redhat.com/mailman/listinfo/freeipa-users" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/freeipa-users</a> > > > >> > Go to <a href="http://freeipa.org" rel="noreferrer" target="_blank">http://freeipa.org</a> for more info on the project > > > >> > > > >> -- > > > >> Manage your subscription for the Freeipa-users mailing list: > > > >> <a href="https://www.redhat.com/mailman/listinfo/freeipa-users" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/freeipa-users</a> > > > >> Go to <a href="http://freeipa.org" rel="noreferrer" target="_blank">http://freeipa.org</a> for more info on the project > > > >> > > > > > > > > > > > > > > > > </div></div></blockquote></div> </div>