<div dir="ltr"><div>Rob,</div><div>The only message that seems remotely relevant is:</div><div> </div><div>ProfileSubmitServlet: for renewal, original authenticator not found</div><div> </div><div>But everything else looks completely fine until the "AUTH_FAIL" message. </div><div>I started seeing</div><div> </div><div>csngen_new_csn - Warning: too much time skew (-xxx secs). Current seqnum=1</div><div> </div><div>So I searched for that and found a few articles...but most of them deal with replication. I don't have any replication agreements right now, and I updated nsslapd-ignore-time-skew to on, but that didn't fix it either.</div><div> </div><div>Any ideas?</div><div> </div><div>Thanks</div></div><div class="gmail_extra"> <div class="gmail_quote">On Mon, Aug 1, 2016 at 3:29 PM, Rob Crittenden <<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Adam Lewis wrote: <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid"> Yup. I'm currently still sitting back in time. But any time I try to resubmit either the ipaCert or the subsystemCert it errors out. getcert list shows : ca-error: Server at "<a href="https://ipa.local.domain:9443/ca/agent/ca/profileProcess" target="_blank" rel="noreferrer">https://ipa.local.domain:9443/ca/agent/ca/profileProcess</a>" replied: 1: Authentication Error And the debug log shows: SignedAuditEventFactory: create() message=[AuditEvent=AUTH_FAIL][SubjectID=$Unidentified$][Outcome=Failure][AuthMgr=certUserDBAuthMgr][AttemptedCred=CN=IPA RA,O=MISS.ION] authentication failure ReviewReqServlet: Invalid Credential. </blockquote> I'd look at the lines above that for clues, and check the 389-ds access log. I assume it is finding an entry for uid=ipara, right? The way the auth works as I understand it is dogtag first compares the serial number, issuer and subject of the provided certificate with the description attribute in the entry it finds in LDAP. Then it compares the full certificate. If things match up then you are authenticated. It then does some authorization work. For reference, mine looks like: dn: uid=ipara,ou=people,o=ipaca objectClass: top objectClass: person objectClass: organizationalPerson objectClass: inetOrgPerson objectClass: cmsuser uid: ipara sn: ipara cn: ipara usertype: agentType userstate: 1 userCertificate:: MIIDbTCCAlWgAwIBAgIBBzANBgkqhkiG9w0BAQsFADA2MRQwEgYDVQQKEwtH [snip] o0i1CCw1v++2tgvHiiZEEeeuOEMGEdXZfv4Xw= description: 2;7;CN=Certificate Authority,O=<a href="http://EXAMPLE.COM" target="_blank" rel="noreferrer">EXAMPLE.COM</a>;CN=IPA RA,O=<a href="http://EXAMPLE.COM" target="_blank" rel="noreferrer">EXAMPLE.COM</a> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid"> Those appear to be the most significant messages. I'm disconnected so getting the full log info is difficult. If it's the only way let me know and I'll see what I can do. Worst case it'll just take me a while to re-type it. </blockquote> Understood. <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid"> Thanks On Mon, Aug 1, 2016 at 3:11 PM, Rob Crittenden <<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>>> wrote: Adam Lewis wrote: Yup, It's just the text string. I don't know how much this matters but when I ran the start-tracking for the ipaCert it didn't generate a new certificate. I'm still working off of serial number 7, which is what it's been since we installed IPA. Is there some way/reason for me to generate a whole new ipaCert? certmonger will take care of that when renewal happens. Did you go back in time to when this cert was valid? rob Thanks On Mon, Aug 1, 2016 at 3:00 PM, Rob Crittenden <<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>>>> wrote: Adam Lewis wrote: If you mean the usercertificate value from the ldapsearch command, then yes. That value matches the value from the certutil output. The usercertificate in LDAP had the BEGIN/END stripped, right? I'll cc a couple of the dogtag developers to see what they think. rob Thanks On Mon, Aug 1, 2016 at 11:18 AM, Rob Crittenden <<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>>> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>>>>> wrote: <div><div class="h5"> Adam Lewis wrote: A quick update. We did some digging on the segfault problem and I think it was due to having to update the trusts on the CA cert. So we updated the certmonger package and certmonger now starts again. However we're kind of back to square one where we are still getting the AUTH_FAIL messages in the debug log. I have verified that the ipara entry's serial number and cert match the serial number and cert from the one in /etc/httpd/alias. How about the certificate PEM? Does it match the usercertificate in the dogtag LDAP server? rob Any other ideas? Thanks! On Mon, Aug 1, 2016 at 9:17 AM, Adam Lewis <<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>>>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>>>>>> wrote: Rob, Thanks for pointing me in the right direction. However after following the instructions in the above mentioned doc I noticed a few things that are odd and have a new problem. The first odd thing I noticed is that when I run service pki-cad status it shows that my PKI Subsystem Type is "CA Clone (Security Domain)" Shouldn't that say something like "CA Master"? Second, when I ran the "ipa-getcert resubmit -I [ID]" commands they all produced the same AUTH_FAIL message in the debug log. Now the new problem...after pressing on and restarting things certmonger fails to start with a segfault. Starting certmonger: /bin/bash: line 1: 64935 Segmentation fault /usr/sbin/certmonger -S -p /var/run certmonger.pid Thanks! On Thu, Jul 28, 2016 at 3:36 PM, Rob Crittenden <<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>>> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>>>> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>>> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a> <mailto:<a href="mailto:rcritten@redhat.com" target="_blank">rcritten@redhat.com</a>>>>>> wrote: Lewis, Adam M CIV NSWCDD, H11 wrote: We are currently dead in the water. Our OCSP, CA Audit, CA Subsystem, and IPA RA certs expired as of 7/23/16. I found and followed the instructions to the letter (<a href="http://www.freeipa.org/page/Howto/Promote_CA_to_Renewal_and_CRL_Master#Procedure_in_FreeIPA_.3C_4.0" target="_blank" rel="noreferrer">http://www.freeipa.org/page/Howto/Promote_CA_to_Renewal_and_CRL_Master#Procedure_in_FreeIPA_.3C_4.0</a>) however the CA Subsystem and IPA RA certs will not renew. I've backdated the server to make sure the system was within the renewal window, but that has not help. Those are the wrong instructions. You want this instead, <a href="https://access.redhat.com/solutions/643753" target="_blank" rel="noreferrer">https://access.redhat.com/solutions/643753</a> A bunch of it is for 2.2 but it isn't exactly noted which parts. A general rule is that you don't/shouldn't need to directly tweak the dogtag configuration or do any of the start-tracking work (though you may want to verify that what/if anything you changed from that wrong doc). When I run getcert list it reports: Ca-error: Sever at "https://<fqdn>:9443/ca/agent/ca/profileProcess" replied: 1: Authentication Error for both the IPA RA and CA Subsystem certs The debug log shows: SignedAuditEventFactory: create() message=[AuditEvent=AUTH_FAIL][SubjectID=$Unidentified$][Outcome=Failure][AuthMgr=certUserDBAuthMgr][AttemptedCred=CN=IPA RA,O=MISS.ION] authentication failure ReviewReqServlet: Invalid Credential. The place to start is to get the serial # of the ipaCert: # certutil -L -d /etc/httpd/alias -n ipaCert |grep Serial Now get the user from the dogtag LDAP server: # ldapsearch -h `hostname` -p 7389 -x -D 'cn=directory manager' -W -b uid=ipara,ou=People,o=ipaca description The format is 2;<serial number>;<issuer subject>;<subject> See if the serial # matches ipaCert. I'm guessing it won't. Follow the instructions on the page I cited to update the entry with the current certificate and serial # values. That should get you going. rob We are kind of in deep doo-doo until this gets resolved. We are running ipa-server-3.0.0-47.el6_7.2 on RHEL 6.5 Any thoughts? Thanks! Adam M. Lewis -- Manage your subscription for the Freeipa-users mailing list: <a href="https://www.redhat.com/mailman/listinfo/freeipa-users" target="_blank" rel="noreferrer">https://www.redhat.com/mailman/listinfo/freeipa-users</a> Go to <a href="http://freeipa.org" target="_blank" rel="noreferrer">http://freeipa.org</a> for more info on the project -- Adam M. Lewis <a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>>>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>> </div></div> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>>>>> 10807 Allie Place Fredericksburg, VA 22408 <a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a>> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a>>> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a>> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a>>>> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a>> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a>>> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a>> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a>>>>> -- Adam M. Lewis <a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>>>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>>>>><div><div class="h5"> 10807 Allie Place Fredericksburg, VA 22408 <a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a>> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a>>> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a>> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a>>>> -- Adam M. Lewis <a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>>>> 10807 Allie Place Fredericksburg, VA 22408 <a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a>> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a>>> -- Adam M. Lewis <a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>>> 10807 Allie Place Fredericksburg, VA 22408 <a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a> <tel:<a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a>> -- Adam M. Lewis <a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> <mailto:<a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a>> 10807 Allie Place Fredericksburg, VA 22408 <a href="tel:540-412-8643" target="_blank" value="+15404128643">540-412-8643</a> </div></div></blockquote> </blockquote></div> -- <div class="gmail_signature" data-smartmail="gmail_signature">Adam M. Lewis <a href="mailto:alewis422@gmail.com" target="_blank">alewis422@gmail.com</a> 10807 Allie Place Fredericksburg, VA 22408 540-412-8643 </div> </div>