<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">I followed Rob's advice and enabled the http debug logging. I'm no expert here but I didn't really see anything that indicates an error. I do see a series of connections to the server and GSS/kerberos log messages (like "client delegates us their credentials" and "GSS-API token of length 22 bytes will be sent back..."). The log just stops and apache returns an authentication error.<div><div><div><br></div><div>While reading the logs, I did notice a reference to s4u2proxy. When I looked around for that, I ran across <a href="http://www.samba.org/~idra/blog/index.html">a blog article from idra at samba.org</a> where he described how they modified mod_auth_kerb for FreeIPA. </div><div><br></div><div>On suspecting something went wrong with the installation (like some update that replaced mod_auth_kerb or something related), I did a "yum reinstall freeipa-server". After it completed, everything started working as it had in the past.</div><div><br></div><div>So while I don't know for sure, I believe some other package that was updated (perhaps apache) overwrote something in FreeIPA (perhaps mod_auth_kerb). In any case, I'm whole again.</div><div><br></div><div>Brian</div><div><br></div><div>On Oct 17, 2012, at 7:49 AM, Rob Crittenden wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>Brian Vetter wrote:<br><blockquote type="cite">I had a happy, working 2.2 FreeIPA installation humming along last<br></blockquote><blockquote type="cite">week. I had to do some maintenance so I shut everything down. When I<br></blockquote><blockquote type="cite">brought everything up, I can no longer log into the web admin tool. I<br></blockquote><blockquote type="cite">get a "Kerberos ticket is no longer valid" error.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">Using the troubleshooting pages on the wiki as a guide, I can kinit<br></blockquote><blockquote type="cite">successfully and see the tickets using klist. I can use the ldapsearch<br></blockquote><blockquote type="cite">tool using GSSAPI to authenticate as well and can return results from<br></blockquote><blockquote type="cite">the ldap server. 'ldapsearch -Y GSSAPI -b "dc=dcc,dc=mobi" uid=admin'<br></blockquote><blockquote type="cite">returns a valid ldap recors for my admin user. I ran this command kinit<br></blockquote><blockquote type="cite">from multiple kerberos principals/users and each worked.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I verified my config settings again with firefox and they are still set<br></blockquote><blockquote type="cite">correctly (auth.delegation-uris, auth.trusted-users both set to my<br></blockquote><blockquote type="cite">domain .dcc.mobi). The cert was accepted (no warnings about the cert not<br></blockquote><blockquote type="cite">being trusted because I had already set it to trusted). I turned on the<br></blockquote><blockquote type="cite">NSPR logging as described in the documents, and didn't see an error,<br></blockquote><blockquote type="cite">although I can't tell if this is correct:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> 492291904[7f4d1d31f6a0]: using REQ_DELEGATE<br></blockquote><blockquote type="cite"> 492291904[7f4d1d31f6a0]: service = geonosis.dcc.mobi<br></blockquote><blockquote type="cite"> 492291904[7f4d1d31f6a0]: using negotiate-gss<br></blockquote><blockquote type="cite"> 492291904[7f4d1d31f6a0]: entering nsAuthGSSAPI::nsAuthGSSAPI()<br></blockquote><blockquote type="cite"> 492291904[7f4d1d31f6a0]: Attempting to load gss functions<br></blockquote><blockquote type="cite"> 492291904[7f4d1d31f6a0]: entering nsAuthGSSAPI::Init()<br></blockquote><blockquote type="cite"> 492291904[7f4d1d31f6a0]: nsHttpNegotiateAuth::GenerateCredentials()<br></blockquote><blockquote type="cite"> [challenge=Negotiate]<br></blockquote><blockquote type="cite"> 492291904[7f4d1d31f6a0]: entering nsAuthGSSAPI::GetNextToken()<br></blockquote><blockquote type="cite"> 492291904[7f4d1d31f6a0]: leaving nsAuthGSSAPI::GetNextToken [rv=0]<br></blockquote><blockquote type="cite"> 492291904[7f4d1d31f6a0]: Sending a token of length 1394<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">There is nothing in /var/log/httpd/error_log. /var/log/httpd/access_log<br></blockquote><blockquote type="cite">does have a few entries:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> 10.1.1.10 - - [16/Oct/2012:18:05:26 -0500] "POST /ca/ocsp HTTP/1.1"<br></blockquote><blockquote type="cite"> 200 2298 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:16.0)<br></blockquote><blockquote type="cite"> Gecko/20100101 Firefox/16.0"<br></blockquote><blockquote type="cite"> 10.1.1.10 - - [16/Oct/2012:18:05:26 -0500] "POST /ipa/session/json<br></blockquote><blockquote type="cite"> HTTP/1.1" 401 -<br></blockquote><blockquote type="cite"> 10.1.1.10 - - [16/Oct/2012:18:05:26 -0500] "GET<br></blockquote><blockquote type="cite"> /ipa/session/login_kerberos HTTP/1.1" 401 1861<br></blockquote><blockquote type="cite"> 10.1.1.10 - <a href="mailto:admin@DCC.MOBI">admin@DCC.MOBI</a><br></blockquote><blockquote type="cite"> [16/Oct/2012:18:05:26 -0500] "GET /ipa/session/login_kerberos<br></blockquote><blockquote type="cite"> HTTP/1.1" 200 -<br></blockquote><blockquote type="cite"> 10.1.1.10 - - [16/Oct/2012:18:05:26 -0500] "POST /ipa/session/json<br></blockquote><blockquote type="cite"> HTTP/1.1" 401 -<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">The 401's aren't surprising here since somehow, something is not<br></blockquote><blockquote type="cite">properly authenticating.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I also looked in /var/log/krb5kdc.log and see the following line when<br></blockquote><blockquote type="cite">authenticating:<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> Oct 16 18:12:17 geonosis.dcc.mobi krb5kdc[1193](info): TGS_REQ (1<br></blockquote><blockquote type="cite"> etypes {18}) 10.1.1.10: ISSUE: authtime 1350424404, etypes {rep=18<br></blockquote><blockquote type="cite"> tkt=18 ses=18}, <a href="mailto:admin@DCC.MOBI">admin@DCC.MOBI</a> for<br></blockquote><blockquote type="cite"> <a href="mailto:krbtgt/DCC.MOBI@DCC.MOBI">krbtgt/DCC.MOBI@DCC.MOBI</a><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">I don't believe this describes an error, but I'm not an expert reading<br></blockquote><blockquote type="cite">that log type.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"> From what I can tell, the problems seem to be between the apache server<br></blockquote><blockquote type="cite">and the browser. Both worked fine together last week. Is there something<br></blockquote><blockquote type="cite">I can turn on in Apache (perhaps in the ipa.conf or auth_kerb.conf<br></blockquote><blockquote type="cite">files) that can help debug this? Or better yet, anyone else seen this<br></blockquote><blockquote type="cite">and have an answer? Is there some key/ticket/etc associated with the<br></blockquote><blockquote type="cite">http server that might be "wrong" now (somehow whacked during the reboot)?<br></blockquote><br>If you set LogLevel debug in /etc/httpd/conf.d/nss.conf and restart the httpd service you'll get a lot of debugging output from ipa and mod_auth_kerb, one of which may provide some pointers.<br><br>rob<br><br></div></blockquote></div><br></div></body></html>