[Freeipa-devel] Provisioning throughput

[Rob Crittenden]

thierry bordaz wrote:
>
>
> On 05/25/2016 08:49 PM, Rob Crittenden wrote:
>> thierry bordaz wrote:
>>>
>>> Hello,
>>>
>>> Thanks for all the feedbacks. I updated the design accordingly and with
>>> additional tests results
>>> (http://www.freeipa.org/page/V4/Performance_Improvements#Proposed_improvements)
>>>
>>> Several improvements can be done, in particular in DS plugins (memberof,
>>> retroCL), but for "easy" benefit provisioning will be done with memberof
>>> disabled followed by fixup.
>>>
>>> It remains some aspects that are not clear to me:
>>>
>>>   * For best performance, DS tuning and provisioning/fixup would
>>>     preferably be done under 'directory manager'
>>>     That means prompting DM password and writing it into temporary file.
>>>     Is that a concern ?
>>>   * Fixup requires that we know the filters matching the provisioned
>>>     entries. For example :
>>>       o (objectClass=inetorgperson)
>>>       o (objectClass=ipausergroup)
>>>       o (objectClass=ipahost)
>>>       o (objectClass=ipahostgroup)
>>>       o (objectClass=ipasudorule)
>>>       o (objectClass=ipahbacrule)
>>>
>>>         The set of objectclass could be hardcode or provided in the
>>>         provisioning CLI option
>>>         What to do if an entry in in the provision file does not match
>>>         any of those filter ? Should it stop without starting the
>>>         provisioning ?
>>>   * The CLI doing the provisioning could be something like 'ipa
>>>     provision <options>' or should it be a separated command e.g.
>>>     ipa-bulk-load ?
>>
>> It depends. There is a migration command now, ipa migrate-ds, that
>> adds records and is impacted by this. There is also the possibility of
>> looping calls to ipa [user|group|etc]-add.
>
> I agree that migration and bulk load can be linked. If migration
> dump/update a set of entries before filling them into a new instance it
> could use bulk load.
> For set loop of ipa <object>-add, I think they add many others direct
> operations (mainly SRCH) before doing the ADD in order to check
> coherency. bulk load looks more straightforward.

I just wonder if some (all) of this could be done manually. Document how 
to turn off memberof, do the import whatever way is appropriate, then 
run the fixup? I'm not sure what you had in mind.

I don't want to think small but do we expect to be importing a slew of 
hosts, sudorules, etc? I guess the potential is there but would it be on 
the same scale as users? If you focus only on users/groups does that 
change the use case at all?

>> Would it be reasonable to require bulk import to be done on an IPA
>> master so we can leverage the ldapi socket?
> Do you mean using ldapi to reduce network latency or automember or
> something else ?

To avoid the DM password issues. ldapi autobinds to DM when the id is root.

rob