[Freeipa-users] Want faster user-add

[Martin Kosek]

On 12/22/2015 04:16 PM, Simo Sorce wrote:
> On Tue, 2015-12-22 at 10:24 +0100, thierry bordaz wrote:
>> On 12/21/2015 05:55 PM, Daryl Fonseca-Holt wrote:
>>> Hi all,
>>>
>>> Environment: RHEL6 with IPA 3.0 at current RedHat level. 64-core 
>>> 256-GB RAM Oracle x4470 M2.
>>>
>>> During our migration from NIS on Solaris 140,000+ accounts will be 
>>> added. After tuning per the guides dbmon.sh shows no roevicts and we 
>>> get high cache hit ratios.
>>>
>>> Per a previous discussion with the list the input is broken down into 
>>> batches of less than 1,000 users and the default IPA group is changed 
>>> before each batch. This helped greatly.
>>>
>>> Adding all the users takes many hours. Initially ipa user-add takes an 
>>> average 2.3 seconds per user but degrades by the time there are 
>>> 140,000 users to an average 6.7 seconds per user.
>>>
>>> In tracing it appears that a significant portion of the time ipa 
>>> user-add takes is not the add itself, it is the query at the end that 
>>> displays the resulting user account. Is there any legit way to prevent 
>>> this query?
>>>
>>> The length of time it takes to migrate is not a big concern. The 
>>> concern is the start of the fall school term when we typically add 
>>> approximately 1,300 accounts per hour during the registration period 
>>> with our current system.
>>>
>>> All suggestions will be appreciated.
>>>
>>> Regards, Daryl
>>>
>> Hi Daryl,
>>
>> I can reproduce similar trend of user-add becoming slower and slower.
>>
>> Now in my tests (etime=7s) the time was spent half by authentication and 
>> half by ADD and MOD (update of ipausers group). I agree there are many 
>> direct SRCH (~10) but they all seems to be rapid.
>>
>> I know that the vast majority of the time is spent in DS schema-compat 
>> plugin. Disabling it, during provisioning, reduce the duration by ~3.
>> Now I do not know if it is a valid option to disable this plugin during 
>> provisioning.
> 
> As long as the schema compat is not needed by users during the
> provisioning, turning it off is fine. All the contents are regenerated
> at startup anyway. So it can be re-enabled and the server restarted
> after the bulk provisioning is done.

+1. When provisioning users via "ipa migrate-ds" command, schema compat is
strongly suggested to be turned off too.