[Freeipa-users] Slow logins on FreeIPA 4.1.2 (F21)

[Jakub Hrozek]

On Mon, Apr 06, 2015 at 08:01:46PM -0500, Dan Mossor wrote:
> On 04/05/2015 12:51 PM, Dmitri Pal wrote:
> >On 04/05/2015 12:10 AM, Dan Mossor wrote:
> >>I've recently deployed a new domain based on 4.1.2 in F21. We've
> >>noticed an issue and can't quite seem to nail it down. The problem is
> >>that logins are taking an inordinate amount of time to complete - the
> >>fastest logon we can get using LDAP credentials is 8 seconds. During
> >>our testing, even logons to the IPA server itself took over 30 seconds
> >>to complete.
> >>
> >>I've narrowed this down to sssd, but that is as far as I can get. When
> >>cranking up debugging for sshd and PAM, I see a minimum 2 second delay
> >>between ssh handing off the authentication request to sssd and the
> >>reply back. The only troubleshooting I've done is with ssh, but the
> >>area that causes the most grief is Apache logins. We configured Apache
> >>to use PAM for auth through IPA, vice directly calling IPA itself.
> >>Logging in to our Redmine site takes users a minimum of 34 seconds to
> >>complete. Following this, a simple webpage containing two hyperlinks
> >>and two small thumbnail images takes over a minute to load on a
> >>gigabit network.
> >>
> >>The *only* thing changed in this environment was the IPA server. We
> >>moved the Redmine from our old network that was using IPA 3.x (F20
> >>branch) to the new one. My initial reaction was that it was the VM
> >>that was hosting Redmine, but we've run these tests against bare metal
> >>machines in the same network and have the same issue. It appears that
> >>sssd is taking a very, very long time to talk to FreeIPA - even on the
> >>IPA server itself.
> >>
> >>However, Kerberos logins into the IPA web GUI are near instantaneous,
> >>while Username/Password logins take more than a few seconds.
> >>
> >>I need to get this solved. My developers don't appreciate the glory
> >>days of XP taking 5 minutes to log into an IIS 2.1 web server on the
> >>local network. I don't have the budget to keep them at the coffee pot
> >>waiting on the network. So, what further information do you need from
> >>me to track this one down?
> >>
> >>Dan
> >>
> >Several tips.
> >Please check your DNS configuration.
> >Such delay is usually caused by the DNS lookups timing out. That means
> >that the servers probably trying to resolve names against an old DNS
> >server that is not around. Look at resolve.conf and make sure only valid
> >DNS servers are there and they are in the proper order.
> >
> >If this does not help please turn on SSSD debug_level to 10, sanitize
> >and send the SSSD domain logs and sssd.conf to the list.
> >More hints can be found here:
> >https://fedorahosted.org/sssd/wiki/Troubleshooting
> >
> DNS lookups are good - 'dig' and 'dig -x' return instantaneous forward and
> reverse lookups on the IPA server, the target server, and the client. The
> only DNS server configured is the IPA server.
> 
> I did catch some sssd logs. I set logging to 0x0450 instead of 10, and I
> didn't have time to compare if any different information was caught. If you
> still need me to specify log level 10 or some other setting, let me know.
> The login that these logs are for took 15.371 seconds (checked via 'time ssh
> danofsatx at yoda.example.lcl exit'
> 
> selinux_child.log: http://fpaste.org/207805/
> sssd_sudo.log: http://fpaste.org/207806/
> sssd_pac.log: http://fpaste.org/207807/
> sssd_pam.log: http://fpaste.org/207808/67775142/
> sssd_nss.log: http://fpaste.org/207809/
> sssd.log: http://fpaste.org/207810/
> sssd_example.lcl.log: http://fpaste.org/207811/36832514/

We've recently found a performance problem in the SELinux code. Can you
check if setting:
    selinux_provider = none
improves the performance anyhow?