[Freeipa-users] Want faster user-add

[thierry bordaz]

On 01/04/2016 01:03 PM, Martin Kosek wrote:
> On 12/22/2015 04:16 PM, Simo Sorce wrote:
>> On Tue, 2015-12-22 at 10:24 +0100, thierry bordaz wrote:
>>> On 12/21/2015 05:55 PM, Daryl Fonseca-Holt wrote:
>>>> Hi all,
>>>>
>>>> Environment: RHEL6 with IPA 3.0 at current RedHat level. 64-core
>>>> 256-GB RAM Oracle x4470 M2.
>>>>
>>>> During our migration from NIS on Solaris 140,000+ accounts will be
>>>> added. After tuning per the guides dbmon.sh shows no roevicts and we
>>>> get high cache hit ratios.
>>>>
>>>> Per a previous discussion with the list the input is broken down into
>>>> batches of less than 1,000 users and the default IPA group is changed
>>>> before each batch. This helped greatly.
>>>>
>>>> Adding all the users takes many hours. Initially ipa user-add takes an
>>>> average 2.3 seconds per user but degrades by the time there are
>>>> 140,000 users to an average 6.7 seconds per user.
>>>>
>>>> In tracing it appears that a significant portion of the time ipa
>>>> user-add takes is not the add itself, it is the query at the end that
>>>> displays the resulting user account. Is there any legit way to prevent
>>>> this query?
>>>>
>>>> The length of time it takes to migrate is not a big concern. The
>>>> concern is the start of the fall school term when we typically add
>>>> approximately 1,300 accounts per hour during the registration period
>>>> with our current system.
>>>>
>>>> All suggestions will be appreciated.
>>>>
>>>> Regards, Daryl
>>>>
>>> Hi Daryl,
>>>
>>> I can reproduce similar trend of user-add becoming slower and slower.
>>>
>>> Now in my tests (etime=7s) the time was spent half by authentication and
>>> half by ADD and MOD (update of ipausers group). I agree there are many
>>> direct SRCH (~10) but they all seems to be rapid.
>>>
>>> I know that the vast majority of the time is spent in DS schema-compat
>>> plugin. Disabling it, during provisioning, reduce the duration by ~3.
>>> Now I do not know if it is a valid option to disable this plugin during
>>> provisioning.
>> As long as the schema compat is not needed by users during the
>> provisioning, turning it off is fine. All the contents are regenerated
>> at startup anyway. So it can be re-enabled and the server restarted
>> after the bulk provisioning is done.
> +1. When provisioning users via "ipa migrate-ds" command, schema compat is
> strongly suggested to be turned off too.
For information, accelerating user-add is investigated under 
https://fedorahosted.org/freeipa/ticket/5448.
Schema-compat has a significant impact on ldap ADD and MOD done during 
user-add. Now appropriate setting of scope of others plugins (dna, 
memberof, uniqueness, uuid...) shows that ADD can be reduced by 10 and 
MOD by 2, this even if schema-compat is still enabled.
So there are also possible improvements in plugin tuning.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/freeipa-users/attachments/20160104/07eb72b0/attachment.htm>