[Freeipa-devel] Provisioning throughput

[thierry bordaz]

On 05/26/2016 09:32 AM, Alexander Bokovoy wrote:
> On Wed, 25 May 2016, Rob Crittenden wrote:
>> thierry bordaz wrote:
>>>
>>>
>>> On 05/25/2016 08:49 PM, Rob Crittenden wrote:
>>>> thierry bordaz wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> Thanks for all the feedbacks. I updated the design accordingly and 
>>>>> with
>>>>> additional tests results
>>>>> (http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements) 
>>>>>
>>>>>
>>>>> Several improvements can be done, in particular in DS plugins 
>>>>> (memberof,
>>>>> retroCL), but for "easy" benefit provisioning will be done with 
>>>>> memberof
>>>>> disabled followed by fixup.
>>>>>
>>>>> It remains some aspects that are not clear to me:
>>>>>
>>>>>  * For best performance, DS tuning and provisioning/fixup would
>>>>>    preferably be done under 'directory manager'
>>>>>    That means prompting DM password and writing it into temporary 
>>>>> file.
>>>>>    Is that a concern ?
>>>>>  * Fixup requires that we know the filters matching the provisioned
>>>>>    entries. For example :
>>>>>      o (objectClass=inetorgperson)
>>>>>      o (objectClass=ipausergroup)
>>>>>      o (objectClass=ipahost)
>>>>>      o (objectClass=ipahostgroup)
>>>>>      o (objectClass=ipasudorule)
>>>>>      o (objectClass=ipahbacrule)
>>>>>
>>>>>        The set of objectclass could be hardcode or provided in the
>>>>>        provisioning CLI option
>>>>>        What to do if an entry in in the provision file does not match
>>>>>        any of those filter ? Should it stop without starting the
>>>>>        provisioning ?
>>>>>  * The CLI doing the provisioning could be something like 'ipa
>>>>>    provision <options>' or should it be a separated command e.g.
>>>>>    ipa-bulk-load ?
>>>>
>>>> It depends. There is a migration command now, ipa migrate-ds, that
>>>> adds records and is impacted by this. There is also the possibility of
>>>> looping calls to ipa [user|group|etc]-add.
>>>
>>> I agree that migration and bulk load can be linked. If migration
>>> dump/update a set of entries before filling them into a new instance it
>>> could use bulk load.
>>> For set loop of ipa <object>-add, I think they add many others direct
>>> operations (mainly SRCH) before doing the ADD in order to check
>>> coherency. bulk load looks more straightforward.
>>
>> I just wonder if some (all) of this could be done manually. Document 
>> how to turn off memberof, do the import whatever way is appropriate, 
>> then run the fixup? I'm not sure what you had in mind.
>>
>> I don't want to think small but do we expect to be importing a slew 
>> of hosts, sudorules, etc? I guess the potential is there but would it 
>> be on the same scale as users? If you focus only on users/groups does 
>> that change the use case at all?
> I tend to agree with Rob on this. Maybe we should have a simple
> script/update file that does preparatory work and another one that
> returns configuration into the right state and document how to use them.

Ok.
rereading the thread I realize we are talking of 
user/usergroups/host/hostgroup.

Provisioning such entries is not that bad.
For example 5Kusers/hosts are provisioned in 5min without memberof and 
19min with memberof

The real problem is provisioning sudorules and hbacrules where the 
impact of memberof is very important.
For example 100 sudorules are provisioned in 30s without memberof and 2h 
with memberof.

Do you think provisioning should also considere sudorules/hbac or only 
user/usergroups/host/hostgroup ?