[Freeipa-devel] Provisioning throughput

[Petr Vobornik]

On 05/04/2016 02:20 PM, thierry bordaz wrote:
> Hello,
> 
>     I have been doing some tests/measures using
>     https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py.
>     The tool creates a set of typical users/hosts/groups... to import with a
>     ldapadd.
> 
>     I wrote down some finding in
>     http://www.freeipa.org/page/V4/Performance_Improvements#Provisioning_throughput_and_DS_plugins.
>     I still have to do some cleanup around the performance but the basic of a
>     possible improvement is to do provisioning in several steps (disabling
>     plugins, provisioning, enabling plugin, running fixup tasks).
> 
>     Before going further in the design I wanted to share those ideas and know if
>     it raise any concern.
> 
>     thanks
>     thierry
> 

Let me conclude the previous sub-thread and also continue with
discussion we had over phone.

The subthread ended with proposal to go with offline provisioning by
disabling LDAP ports similar to ipa-server-upgrade tool and then using
LDAPI to add records by using ipalib in server mode e.g., as in
ipa_otptoken_import.py

So these are the tasks to solve/investigate:

1. Provide guidance/write script which would disable memberof plugin and
other plugins. Disable ldap and ldaps port

2. Provide guidance/script how to use ipalib in server mode and how to
import date. This could be even script which would load file in some
format(e.g. JSON) and executed commands from the file. Basically what
was Alexander proposing and I was against it. After some thought, I
agree that the tool could be easy to write but I'd rather avoid adding
it to 4.4 release maybe even to future releases. Because:
 - It's almost the same as `[RFE] run multiple CLI commands in a batch`
<https://fedorahosted.org/freeipa/ticket/5821> With the distinction that
it connects directly LDAPI and not public API. First I'd like to see
#5821 and then we can think of using same logic(input) to work in
"migration" mode.
- I don't want to include a quickly baked tool so late in 4.4
development. It will have design flaws which will be harder to fix
later. I don't want to loose time with discussion about design of the
tool in this phase of 4.4.

3. Provide guidance/script to revert #1 and run memberof fixup task.

4. Investigate how replication of so many records with member attributes
will affect other replicas. I.e. if it would not brick entire topology.

*Right now #4 seems to be the most important*.

Then #1, #2, #3 could be delivered as sample script included in
documentation and/or blog post/github. It would allow us to change it
later. I would not focus on #2 before other core 4.4 items are finished.
A lot of guidance is already written in tree design page but it will
need to be transform to easily consumable form for admins. The scripts
can be prepared even when 4.4 is out.

In other words I would not create any official new ipa utility yet.
-- 
Petr Vobornik