[Freeipa-users] ipa-replica-install IPA startup timing issue
thierry bordaz
tbordaz at redhat.com
Mon Mar 14 14:44:59 UTC 2016
Hi Daryl,
In fact the slow DS startup is due to slapi-nis priming:
#0 0x00007f189a2689fc in strcmpi_fast
#1 oc_find_nolock
#2 0x00007f189a2699bd in va_expand_one_oc
#3 0x00007f189a269d70 in schema_expand_objectclasses_ext
#4 0x00007f189a26cbea in slapi_schema_expand_objectclasses
#5 0x00007f189a2124a5 in slapi_str2entry
#6 0x00007f188c38037d in backend_set_entry_from
#7 0x00007f188c383316 in backend_shr_set_entry_cb
#8 0x00007f189a26358d in send_ldap_search_entry_ext
#9 0x00007f189a263dcc in send_ldap_search_entry
#10 0x00007f189a240ad3 in iterate
#11 0x00007f189a240c7a in send_results_ext
#12 0x00007f189a24265e in op_shared_search
#13 0x00007f189a2528de in search_internal_callback_pb
#14 0x00007f188c387628 in backend_shr_set_config_entry_add
#15 0x00007f188c3827ad in backend_set_config_entry_add_cb
#16 0x00007f189a26358d in send_ldap_search_entry_ext
#17 0x00007f189a263dcc in send_ldap_search_entry
#18 0x00007f189a240ad3 in iterate
#19 0x00007f189a240c7a in send_results_ext
#20 0x00007f189a24265e in op_shared_search
#21 0x00007f189a2528de in search_internal_callback_pb
#22 0x00007f188c387cbb in backend_shr_startup
#23 0x00007f188c394135 in plugin_startup
#24 0x00007f189a24d847 in plugin_call_func
#25 0x00007f189a24df78 in plugin_call_one
#26 plugin_dependency_startall
#27 0x00007f189a24e381 in plugin_startall
#28 0x00007f189a716bc2 in main
It lasts from Mon Mar 14 08:50:21 -> Mon Mar 14 08:51:17 CDT
kadmin.service failed to start but the console log does not contain the
exact time of the failure.
Would you check if the failure occurred while DS was starting up ?
If that is the case, like Alexander mentioned, it is already fixed in
slapi-nis 0.55.
thanks
thierry
On 03/14/2016 03:06 PM, Daryl Fonseca-Holt wrote:
> Hi Thierry,
>
> I moved the old logs into a subdirectory called try1. I did the
> recommended ipa-server-install --uninstall. Tried the replica install
> again. Failed during kadmind start like the previous time.
>
> The log from ipa-replica-install (with -d) is at
> http://home.cc.umanitoba.ca/~fonsecah/ipa/ipareplica-install.log
> The console script (mostly the same as the log but with my entries) is
> at http://home.cc.umanitoba.ca/~fonsecah/ipa/ipa-replica-install.console
> The 5 second pstacks are at
> http://home.cc.umanitoba.ca/~fonsecah/ipa/slapd-pstacks.console
>
> Thanks, Daryl
>
>
> On 03/11/16 02:40, thierry bordaz wrote:
>> Hello Deryl,
>>
>> My understanding is that ns-slapd is first slow to startup. Then
>> when krb5kdc is starting it may load ns-slapd.
>>
>> We identified krb5kdc may be impacted by the number of users
>> accounts.
>> From the ns-slapd errors log it is not clear why it is so slow to
>> start.
>>
>> Would you provide the ns-slapd access logs from that period.
>> Also in order to know where ns-slapd is spending time, it would
>> really help if you can get regular (each 5s) pstacks (with
>> 389-ds-debuginfo), during DS startup and then later during
>> krb5kdc startup.
>>
>> best regards
>> thierry
>>
>>
>> On 03/10/2016 11:10 PM, Daryl Fonseca-Holt wrote:
>>> Environment:
>>> RHEL 7.2
>>> IPA 4.2.0-15
>>> nss 3.19.1-19
>>> 389-ds-base 1.3.4.0-26
>>> sssd 1.13.0-40
>>>
>>>
>>> I've encountered this problem in IPA 3.0.0 but hoped it was
>>> addressed in 4.2.0.
>>>
>>> Trying to set up a replica of a master with 150,000+ user accounts,
>>> NIS and Schema Compatability enabled on the master.
>>>
>>> During ipa-replica-install it attempts to start IPA. dirsrv starts,
>>> krb5kdc starts, but then kadmind fails because krb5kdc has gone
>>> missing.
>>>
>>> This happens during restart of IPA in version 3.0.0 too. There it
>>> can be overcome by manually starting each component of IPA _but_
>>> waiting until ns-slapd-<instance> has settled down (as seen from
>>> top) before starting krb5kdc. I also think that the startup of
>>> krb5kdc loads the LDAP instance quite a bit.
>>>
>>> There is a problem in the startup logic where dirsrv is so busy that
>>> even though krb5kdc successfully starts and allows the kadmin to
>>> begin kdb5kdc is not really able to do its duties.
>>>
>>> I'm reporting this since there must be some way to delay the start
>>> of krb5kdc and then kadmind until ns-slapd-<instance> is really open
>>> for business.
>>>
>>> # systemctl status krb5kdc.service
>>> ● krb5kdc.service - Kerberos 5 KDC
>>> Loaded: loaded (/usr/lib/systemd/system/krb5kdc.service;
>>> disabled; vendor preset: disabled)
>>> Active: inactive (dead)
>>>
>>> Mar 10 14:19:13 jutta.cc.umanitoba.ca systemd[1]: Stopped Kerberos 5
>>> KDC.
>>> Mar 10 14:20:36 jutta.cc.umanitoba.ca systemd[1]: Starting Kerberos
>>> 5 KDC...
>>> Mar 10 14:20:39 jutta.cc.umanitoba.ca systemd[1]: Started Kerberos 5
>>> KDC.
>>>
>>> # systemctl status krb5kdc.service
>>> ● krb5kdc.service - Kerberos 5 KDC
>>> Loaded: loaded (/usr/lib/systemd/system/krb5kdc.service;
>>> disabled; vendor preset: disabled)
>>> Active: inactive (dead)
>>>
>>> Mar 10 14:19:13 jutta.cc.umanitoba.ca systemd[1]: Stopped Kerberos 5
>>> KDC.
>>> Mar 10 14:20:36 jutta.cc.umanitoba.ca systemd[1]: Starting Kerberos
>>> 5 KDC...
>>> Mar 10 14:20:39 jutta.cc.umanitoba.ca systemd[1]: Started Kerberos 5
>>> KDC.
>>>
>>> journalctl -xe was stale by the time I got to it so I've attached
>>> /var/log/messages instead.
>>>
>>> The log from ipa-replica-install (with -d) is at
>>> http://home.cc.umanitoba.ca/~fonsecah/ipa/ipareplica-install.log
>>> The console script (mostly the same as the log but with my entries)
>>> is at
>>> http://home.cc.umanitoba.ca/~fonsecah/ipa/ipa-replica-install.console
>>> The /var/log/dirsrv/ns-slapd-<instance> access log is at
>>> http://home.cc.umanitoba.ca/~fonsecah/ipa/access
>>>
>>> Regards, Daryl
>>>
>>>
>>>
>>
>
> --
> --
> Daryl Fonseca-Holt
> IST/CNS/Unix Server Team
> University of Manitoba
> 204.480.1079
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/freeipa-users/attachments/20160314/28c2ecaa/attachment.htm>
More information about the Freeipa-users
mailing list