[Freeipa-users] Slow ipa performance -- why so many ldap lookups ?

[Jakub Hrozek]

On Tue, Mar 19, 2013 at 09:41:23PM +0100, Jan-Frode Myklebust wrote:

Hello Jan,
I'm sorry you're seeing performance problems. 

> We're struggeling with the performance of IPA, and have tried switching
> to the ldap backend for sssd to be able to see what's happening. The
> attached trace is from a RHEL6.4 client running "id janfrode" with the
> following sssd backend:
> 

You should really use the ipa backend for better performance as it uses
the memberof attribute (and a couple of other shortcuts to be able to
tell if a missing member is a user or a group based on the format of the
DN for example).

> -----------------------------------------------------------------------
> [domain/IPALDAP]
> id_provider = ldap
> auth_provider = ldap
> ldap_schema = rfc2307bis
> ldap_uri = ldap://ipa2.example.net, ldap://ipa1.example.net
> ldap_search_base = dc=example,dc=net
> ldap_user_search_base = cn=users,cn=accounts,dc=example,dc=net
> ldap_group_search_base = cn=groups,cn=accounts,dc=example,dc=net
> ldap_netgroup_search_base = cn=ng,cn=compat,dc=example,dc=net
> ldap_tls_cacert = /etc/ipa/ca.crt
> ldap_tls_reqcert = never
> cache_credentials = True
> enumerate = false
> debug_level = 1
> krb5_server = ipa1.example.net
> -----------------------------------------------------------------------
> 
> Please ignore the timings in the trace, as the ldap-server had too high
> debug level when it was collected.
> 
> What we find very strange in the trace is:
> 
> 	- how many ldap searches are done (144!)

The number really depends on the group structure and nesting levels.

> 	- that nesting is handled by the client, instead of using
> 	  "memberOf".

I'm sorry, I don't quite understand the problem here. If ipa backend was
used, then all groups would be resolved in a single search by fetching
the objects the memberof attribute points at.

On the other hand, the RFC2307bis schema does not guarantee there is a
memberof attribute at all, so the client has to perform multiple queries
based on the member attribute. This is one of the prime reasons to stick to
the ipa backend as opposed to the LDAP back end with the RFC2307bis schema.

> 	- that all group members are searched individually, and multiple
> 	  times if they're members of multiple groups
> 

They shouldn't be fetched multiple times, sounds like a bug to me. How
did you measure this metric? Wireshark lookups? Or just analysis of the
logs?

> Ldap-searches are probably (without this high debug level) taking
> 0.02-0.03s, so 144 of these is causing us 3-4s to resolve all my groups.
> 
> I can get the time "id janfrode" takes down to less than 1s if I use
> ldap_schema=rfc2307, turn of nesting and use the cn=compat tree for
> group lookups... but ideally I'd want to use the ipa-backend so that we
> can use the IPA HBAC stuff..

Yes, I would strongly advise against using the compat tree. Let's first
take a look if there's maybe a way to optimize the lookups with the ipa
back end.

Can you tell us a little bit about your nesting structure? How many
users, how many groups, how deep is the nesting?

By the way, the "id" command is not really a fair benchmark as, contrary
to the initgroups() operation that happens during a login, also fetches
all the group members. If you are seeing slow logins, then the best way
to benchmark the initgroups is "id -G", not "id".