[Freeipa-users] Large slow down when using IPA

[Jakub Hrozek]

On Mon, Jan 02, 2012 at 12:53:29PM -0500, Simo Sorce wrote:
> On Mon, 2012-01-02 at 17:29 +0100, Jakub Hrozek wrote:
> > On Mon, Jan 02, 2012 at 10:00:02AM -0500, Simo Sorce wrote:
> > > On Sat, 2011-12-31 at 01:35 -0900, Erinn Looney-Triggs wrote:
> > > > On 12/30/2011 07:19 PM, JR Aquino wrote:
> > > > > 
> > > > > On Dec 30, 2011, at 5:45 PM, Erinn Looney-Triggs wrote:
> > > > > 
> > > > >> I have been slowly rolling out FreeIPA to my systems, trying to track
> > > > >> differences/changes. One of the most noticeable has been a large slow
> > > > >> down in file access times.
> > > > >>
> > > > >> Let me explain as best as I can. I use AIDE to track the file system
> > > > >> (think tripwire) and it runs checks once a day. During these checks it
> > > > >> is scanning (almost) the entire file system and comparing it to a stored
> > > > >> database. On a moderately powered system with ~151k files, an AIDE run
> > > > >> will usually take ~30 minutes. After the system becomes an IPA client
> > > > >> the same run will generally take ~90-120 minutes. Un-install the
> > > > >> ipa-client, back to ~30 minutes for an AIDE run.
> > > > >>
> > > > >> Now clearly a lot of lookups are being done for user names and group
> > > > >> names, and this will have a performance hit that is dependant on the
> > > > >> network. However, the odd thing is that even when running on the IPA
> > > > >> server itself the slowdown is still the same.
> > > > >>
> > > > >> Not sure if this is an IPA problem, an SSSD problem, a bit of both, or
> > > > >> neither, perhaps it is just the way it is, but a slowdown of 3-4x seems
> > > > >> a bit much to me. Clearly the results are not scientific, however, they
> > > > >> have been generally reproducible since I started rolling IPA out.
> > > > >>
> > > > >> As a side note this slowdown has also broken bacula backups, as the
> > > > >> bacula client is scanning the filesystem for change (using accurate
> > > > >> backups) the director times out.
> > > > >>
> > > > >> Any thoughts, or opinions? Workarounds etc? I have checked to make sure
> > > > >> that SSSD caching is enabled, and functional.
> > > > >>
> > > > >> Thanks,
> > > > >>
> > > > >> -Erinn
> > > > > 
> > > > > I am assuming that these are all running as local users.
> > > > > 
> > > > > From the sssd.conf man page in the nss section:
> > > > > 
> > > > > filter_users, filter_groups (string)
> > > > >            Exclude certain users from being fetched from the sss NSS database. This is particularly useful for system accounts. This option can also be set per-domain or include fully-qualified names to filter only users from the
> > > > >            particular domain.
> > > > > 
> > > > >            Default: root
> > > > > 
> > > > > 
> > > > > Try adding this to your sssd.conf:
> > > > > 
> > > > > [nss]
> > > > >            filter_groups = root,bacula,aide,otherdaemonuser <-as needed
> > > > >            filter_users = root,bacula,aide,otherdaemonuser <- as needed
> > > > > 
> > > > > Let me know if that solves your issue.
> > > > > 
> > > > 
> > > > Thanks for pointing that out, completely missed that option! Wouldn't it
> > > > be sweet to have an option that say looked at /etc/login.defs and just
> > > > didn't lookup anything under MIN_UID, on the assumption that those are
> > > > system accounts? Certainly would stop a lot of lookups I imagine.
> > > 
> > > We already have range options (min_id/max_id), but unfortunately that
> > > doesn't help when an application asks for information by name.
> > > You either permanently blacklist such name or you have to do the lookup
> > > and then find out it either a) does not exist, or b) it has to be
> > > filtered out.
> > > 
> > > > Of course you would have to leave it as an option and probably default
> > > > it to off given the odd things people do with their systems.
> > > 
> > > Indeed sssd used to enforce a min id range of 1000 and we turned it off
> > > in the default configuration due to issues with weird configurations.
> > > 
> > > Can you try using both min_id and filter_users and see if it makes any
> > > difference in your case ?
> > > 
> > > Simo.
> > > 
> > 
> > Even when performing getpwuid() calls, SSSD first looks up the user
> > information, reads the UID LDAP attribute and then checks the UID value
> > from LDAP against min_id/max_id values.
> 
> Not according to my reading of the sources, if you look into
> nss_cmd_getpwuid_search() you'll see that we proceed only if we first
> pass the min_id/max_id range check, otherwise we return ENOENT.
> 
> Simo.

Sorry, you're right and I need to warm up my brain a little more after
the Christmas break.

Thanks!