[Freeipa-devel] Provisioning throughput

[thierry bordaz]

On 05/05/2016 03:44 PM, Petr Vobornik wrote:
> On 05/04/2016 02:20 PM, thierry bordaz wrote:
>> Hello,
>>
>>      I have been doing some tests/measures using
>>      https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.
>>      The tool creates a set of typical users/hosts/groups... to import with a
>>      ldapadd.
>>
>>      I wrote down some finding in
>>      http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.
>>      I still have to do some cleanup around the performance but the basic of a
>>      possible improvement is to do provisioning in several steps (disabling
>>      plugins, provisioning, enabling plugin, running fixup tasks).
>>
>>      Before going further in the design I wanted to share those ideas and know if
>>      it raise any concern.
>>
>>      thanks
>>      thierry
>>
> Hi Thierry,
>
> Thanks for the analysis. Very nice.
>
> Knowing this will help us suggesting workarounds also for old releases.
>
> Couple questions:
>
> Have you tested retrCL disabled with memberOf enabled. It seems that it
> would eliminate 550K adds and 0.8M searches. What would be the time
> improvement?
>
> Do you know what is the time when memberof is enabled but slapi-nis and
> retroCL are disabled?
The culprit of the performance issue is very likely related to SRCH 
(internal) triggered by memberof.

If retroCL is disabled and memberof enabled, #SRCH is 13.8M.
If retroCL is disabled, slapi-nis disabled and memberof enabled #SRCH is 
14.8
When all of them are enabled the #SRCH is 15M.

You are right if retroCL is disabled the #ADD drops but it has no 
significant effect on the duration.

Regarding the duration of the provisioning, values are not really stable 
as performance of VM fluctuates. But as soon as memberof is enabled the 
provisioning lasts > 4hours where the same provisioning lasts 6mins as 
soon as memberof is disabled.

I need to confirm the average time for internal searches but assuming 
1ms per SRCH it consumes >90% of the provisioning.


>
>  From the text it was not clear to me, if you find or investigate
> possible improvements in memberof plugin which would improve the
> performance without stopping and starting DS.