[Freeipa-devel] [PATCH] script for provisioning

[thierry bordaz]

On 06/05/2016 10:45 AM, Martin Basti wrote:
>
>
>
> On 03.06.2016 17:49, thierry bordaz wrote:
>> Hello,
>>
>> A performance bottleneck during provisioning was described 
>> http://www.freeipa.org/page/V4/Performance_Improvements#typical_provisioning:_ldapadd_entries.2C_migrate-ds...
>>
>> I wrote the attached script that is following 
>> http://www.freeipa.org/page/V4/Performance_Improvements#Algorithm
>>
>> This is a preliminary script that needs to be improved but just to 
>> check it matches basic requirements I am sending it to the alias to 
>> get some feedback.
>>
>> The output of a provisioning session is below. The steps are:
>>
>>   * install freeipa-server
>>   * create a provisioning file using
>>     https://github.com/freeipa/freeipa-tools/blob/master/create-test-data.py
>>   * ipa-provision.py prepare
>>   * ipa-provision.py import
>>   * ipa-provision.py finish
>>
>> Note: you will likely need to define the DM password with the option 
>> '-w <password>'
>>
>> regards
>> thierry
>>
>>
>> /tmp/ipa-provision.py prepare -f 
>> /tmp/1K_users_800_hosts_groups_hostgroups_sudorules_hbac_rules.ldif -d 1
>> entrycache = 10485760
>> dncache    = 10485760
>> dbcache    = 209715200
>> dblock     = 10000
>> Preparation of the bulk import is now completed
>> If you want to continue:
>>   - run /tmp/ipa-provision.py import -f 
>> /tmp/1K_users_800_hosts_groups_hostgroups_sudorules_hbac_rules.ldif
>>   - run /tmp/ipa-provision.py finish -b <suffix_dn>
>>
>> If you want to not continue:
>>   - run /tmp/ipa-provision.py abort
>>
>>
>>
>> /tmp/ipa-provision import -f 
>> /tmp/1K_users_800_hosts_groups_hostgroups_sudorules_hbac_rules.ldif 
>> -debug 2
>> adding new entry 
>> "uid=user0,cn=users,cn=accounts,dc=abc,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com"
>>
>> adding new entry 
>> "uid=user1,cn=users,cn=accounts,dc=abc,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com"
>>
>> adding new entry 
>> "uid=user2,cn=users,cn=accounts,dc=abc,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com"
>> ...
>> adding new entry 
>> "ipaUniqueID=autogenerate,cn=hbac,dc=abc,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com"
>>
>> adding new entry 
>> "ipaUniqueID=autogenerate,cn=hbac,dc=abc,dc=idm,dc=lab,dc=eng,dc=brq,dc=redhat,dc=com"
>>
>> Bulk import is now completed
>> You need to run:
>>   - run /tmp/ipa-provision.py finish -b <suffix_dn>
>>
>>
>>
>> /tmp/ipa-provision.py finish
>>
>> Waiting for (objectClass=inetorgperson) fixup to complete...
>> Completed filter (objectClass=inetorgperson)
>>
>> Waiting for (objectClass=ipausergroup) fixup to complete...
>> Completed filter (objectClass=ipausergroup)
>>
>> Waiting for (objectClass=ipahost) fixup to complete...
>> Completed filter (objectClass=ipahost)
>>
>> Waiting for (objectClass=ipahostgroup) fixup to complete...
>> Completed filter (objectClass=ipahostgroup)
>>
>> Waiting for (objectClass=ipasudorule) fixup to complete...
>> Completed filter (objectClass=ipasudorule)
>>
>> Waiting for (objectClass=ipahbacrule) fixup to complete...
>> Completed filter (objectClass=ipahbacrule)
>>
>> Bulk import and fixup tasks are now completed
>>
>>
>>
>
> Thank you for provisioning script:
>
> * should be this part of freeIPA or is it just script for testing 
> purposes?
> * the code isn't pythonic  much, if it is just for testing I don't 
> care, but if this will be part of freeIPA I care a lot (I can help in 
> this case :) )
> * maybe some parts of IPA code can be reused for this script, is has 
> to run on installed server anyway
>
> Martin^2
>
>
Hi Martin,

Thanks for your review of that script.. over the weekend :-)
It should be part of freeipa as a way to support provisioning of large 
number of entries.
I am not really surprised it is not pythonic, I tried first to use IPA 
code but got lost and finally decided to do it that easy way.
I am fine if it needs to be rewritten and would really appreciate your 
help :-)

Yes it has to run on an installed server. But even being part of Freeipa 
deliver, this script would preferably run with freeipa being stop and 
only DS running. It saves memory and limits DS ops.

Thanks
thierry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20160606/f7bdeaa5/attachment.htm>