[Freeipa-devel] [PATCHES] 143-147 Improve performance with large groups

[Martin Kosek]

On 06/27/2013 04:55 PM, Jan Cholasta wrote:
> Hi,
> 
> the attached patches are an attempt to solve
> <https://fedorahosted.org/freeipa/ticket/3706> without actually removing ipausers.
> 
> I have done some basic timing on IPA with 10k users, the results are:
> 
>     ipa user-add: 18 s originally, 4 s with the patches
>     ipa user-del: 54 s originally, 7 s with the patches
> 
> Other commands should be affected as well, especially del commands (deleting an
> entry triggers a originally unindexed search in the referint plugin) and member
> manipulation commands (full member list is no longer fetched and stored back
> when adding/removing members).
> 
> Patch 147 fixes <https://fedorahosted.org/freeipa/ticket/3743>.
> 
> Honza
> 

Thanks for this effort!

I quickly went through the patches, they mostly look harmless. Except the
following:

Subject: [PATCH 4/5] Add missing substring indices for attributes managed by
 the referint plugin.

AFAIK, sub index is a very expensive index - as we discussed offline - adding
Rich to advise and confirm this. I think you added it because some plugin was
doing substring/wildcard search when an LDAP entry was being  deleted - did you
identify which one it is? Because I would rather get rid of the bad search than
adding so many sub indices.

Secondly, did you also check Web UI performance? I think we could noticeable
improve user/group lists performance if we added a new (hidden) option to
suppress loading membership information which could then be utilized by Web UI.
Adding Petr Vobornik to CC to consider this.

Martin