[Freeipa-devel] Provisioning throughput

[thierry bordaz]

On 05/05/2016 03:44 PM, Petr Vobornik wrote:
> On 05/04/2016 02:20 PM, thierry bordaz wrote:
>> Hello,
>>
>>      I have been doing some tests/measures using
>>      https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.
>>      The tool creates a set of typical users/hosts/groups... to import with a
>>      ldapadd.
>>
>>      I wrote down some finding in
>>      http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.
>>      I still have to do some cleanup around the performance but the basic of a
>>      possible improvement is to do provisioning in several steps (disabling
>>      plugins, provisioning, enabling plugin, running fixup tasks).
>>
>>      Before going further in the design I wanted to share those ideas and know if
>>      it raise any concern.
>>
>>      thanks
>>      thierry
>>
> Hi Thierry,
>
> Thanks for the analysis. Very nice.
>
> Knowing this will help us suggesting workarounds also for old releases.
>
> Couple questions:
>
> Have you tested retrCL disabled with memberOf enabled. It seems that it
> would eliminate 550K adds and 0.8M searches. What would be the time
> improvement?

Basically provisioning/migrate does not scale well because of memberOf.

Each time you add a member to a group you need to reevalute its 
membership (15M search) and finally update the member (that is a MOD for 
the member + ADD in retroCL). If a same entry is added to a group and 
later added to an other group, the first evaluation/updates are useless.

Disabling memberof drops SRCH from 15M -> ~1M. You can get an additional 
gain ~1M -> ~0.1M if you disable retroCL.

I tested retroCL disabled / memberOf enabled but on a different dataset. 
It gaves low improvement in term of duration.
I think MODs are here more expensive that ADD.
I will test your suggestion to confirm the numbers.


>
> Do you know what is the time when memberof is enabled but slapi-nis and
> retroCL are disabled?

No. I will test it.
>
>  From the text it was not clear to me, if you find or investigate
> possible improvements in memberof plugin which would improve the
> performance without stopping and starting DS.
memberof does not scale well because of #SRCH and #MOD.
Ludwig suggested a group caching that is looking close to other 
in-memory implementations of memberof.
Such cache would reduce the #SRCH possibly from 15M->1M, but the 
implementation is not immediate.
Also note that filling the cache has a cost that can delay startup (like 
we saw in slapi-nis) and a memory footprint.

Regarding the MODs we can make memberof attribute a virtual attribute 
and stop updating the entries. This is also not immediate to implement: 
rewrite memberof plugin.

 From 7.3 schedule, I think we can get an important improvement if DS 
start-stop is acceptable.
Improving memberof is more a longer term option.

thanks
thierry