[Freeipa-users] caching of lookups / performance problem

[Sullivan, Daniel [CRI]]

Hi,

I have another question about sssd performance.  I’m having a difficult time doing a regularly performant ‘ls -l’ operation against /home, a mounted NFS share of all of our users home directories.  There are 667 entries in this folder, and all of them have IDs that are resolvable via freeipa/sssd.  We are using an AD trusted domain.

It is clear to me why an initial invocation of this lookup should take some time (populating the local ldb cache).   And it does.  Usually around 5-10 minutes, but sometimes longer.  After the initial lookups are complete, the output of ‘ls -l' renders fine, and I can inspect the local filesystem cache using ldbsearch and see that it is populated.  The issue is that if I wait a while, or restart sssd, it appears that I have to go through all of these lookups again to render the directory listing.

I am trying to find an optimal configuration for sssd.conf that will allow a performant ‘ls -l’ listing of a directory with a large number of different id numbers assigned to filesystem objects to always return results immediately from the local cache (after an initial invocation of this command for any given directory).  I think basically what I want is to have the ldb cache always ‘up-to-date’, or at least have sssd willing to immediately dump what it has without having to do a bunch of lookups while blocking the ‘ls -l’ thread.  If possible, whatever solution implemented should also survive a restart of the sssd process.  In short, aside from an initial invocation, I never want ‘ls -l’ to take more than a few seconds.

The issue described above is somewhat problematic because it appears to cause contention on the sssd process effectively allowing a user doing ls -l /home to inadvertently degrade system performance for another user.

So far I have tried:

1)  Implementing 'enumeration = true' for the [domain] section .  This seems to have no impact.  It might be worthwhile to note that we are using an AD trusted domain.
2)  Using the refresh_expired_interval configuration for the [domain] section

I have read the following two documents in a decent level of detail:

https://jhrozek.wordpress.com/2015/08/19/performance-tuning-sssd-for-large-ipa-ad-trust-deployments/
https://jhrozek.wordpress.com/2015/03/11/anatomy-of-sssd-user-lookup/

It almost seems to me like the answer to this would be to keep the LDB cache valid indefinitely (step 4 on https://jhrozek.wordpress.com/2015/03/11/anatomy-of-sssd-user-lookup/).

Presumably this is a problem that somebody has seen before.  Would somebody be able to advise on the best way to deal with this?  I appreciate your help.

Thank you,

Dan