[Freeipa-users] Dogtag not working?

Tue Dec 3 06:43:48 UTC 2013

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 12/02/2013 10:18 AM, Rob Crittenden wrote:
> Erinn Looney-Triggs wrote:
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>> 
>> On 12/02/2013 08:03 AM, Rob Crittenden wrote:
>>> Erinn Looney-Triggs wrote:
>>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>>>> 
>>>> On 12/02/2013 07:40 AM, Rob Crittenden wrote:
>>>>> Erinn Looney-Triggs wrote:
>>>>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>>>>>> 
>>>>>> On 11/28/2013 03:50 PM, Erinn Looney-Triggs wrote:
>>>>>>> In the process of prepping a replication host for
>>>>>>> changing over the CA I had to use certmonger to
>>>>>>> generate another certificate on my secondary IPA
>>>>>>> server. Unfortunately it seems to fail every single
>>>>>>> time. Here is what I am running and here is what I am
>>>>>>> getting:
>>>>>>> 
>>>>>>> ipa-getcert request -k private/ipa2.abaqis.com.key -f 
>>>>>>> certs/ipa2.abaqis.com.crt -g 2048
>>>>>>> 
>>>>>>> The request appears to work, however when checking the
>>>>>>> list I receive the following:
>>>>>>> 
>>>>>>> ipa-getcert list -r Number of certificates and
>>>>>>> requests being tracked: 9. Request ID '20131128202128':
>>>>>>> status: CA_UNREACHABLE ca-error: Server failed request,
>>>>>>> will retry: 4301 (RPC failed at server.  Certificate
>>>>>>> operation cannot be completed: FAILURE (Authentication
>>>>>>> Error)). stuck: yes key pair storage: 
>>>>>>> type=FILE,location='/etc/pki/tls/private/ipa2.abaqis.com.key'
>>>>>>>
>>>>>>>
>>
>>>>>>> 
certificate:
>>>>>>> type=FILE,location='/etc/pki/tls/certs/ipa2.abaqis.com.crt'
>>>>>>>
>>>>>>>
>>
>>>>>>> 
CA: IPA issuer: subject: expires: unknown pre-save command:
>>>>>>> post-save command: track: yes auto-renew: yes
>>>>>>> 
>>>>>>> Fine, I check the http logs and get about the same:
>>>>>>> [Thu Nov 28 22:03:06 2013] [error] ipa: ERROR: 
>>>>>>> ipaserver.plugins.dogtag.ra.request_certificate():
>>>>>>> FAILURE (Authentication Error)
>>>>>>> 
>>>>>>> Now as I understand it ipa-getcert is going to
>>>>>>> theserver listed in /etc/ipa/default.conf, which in
>>>>>>> this case is ipa2.abaqis.com (the request is coming
>>>>>>> from the same host). The host principle in
>>>>>>> /etc/krb5.keytab is used for authentication.
>>>>>>> 
>>>>>>> I have tested against the primary ipa server and 
>>>>>>> everything works as it should. However, any requests
>>>>>>> going against ipa2 for certificates are failing.
>>>>>>> 
>>>>>>> At this point I am stuck, so any suggestions are
>>>>>>> welcome.
>>>>>>> 
>>>>>>> -Erinn
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> Replying to myself here, and narrowing this down a bit 
>>>>>> further this seems to be a straight auth problem against
>>>>>> my secondary ipa server. All command work against the
>>>>>> primary, all certificate commands against the secondary
>>>>>> fail.
>>>>>> 
>>>>>> It appears to be confined to dogtag (other commands like
>>>>>> ipa user-show work), but how exactly dogtag handles auth
>>>>>> I am not clear on. It appears as though mod_auth_kerb
>>>>>> handles most things and that is definitely working.
>>>>>> However any access against dogtag components is failing,
>>>>>> so dogtag must/should/may be handling auth internally in
>>>>>> a way that is failing.
>>>>>> 
>>>>>> Anyway, suggestions are still welcome,
>>>>> 
>>>>> Run this on the replica and see if it is being tracked by 
>>>>> certmonger
>>>>> 
>>>>> # getcert list -d /etc/httpd/alias -n ipaCert
>>>>> 
>>>>> If not, see if the a cert with the nickname ipaCert is in 
>>>>> /etc/httpd/alias:
>>>>> 
>>>>> # certutil -L -d /etc/httpd/alias -n ipaCert
>>>>> 
>>>>> If so, see if you have the key:
>>>>> 
>>>>> # certutil -K -d /etc/httpd/alias -n ipaCert -f 
>>>>> /etc/httpd/alias/pwdfile.txt
>>>>> 
>>>>> This is the RA agent certificate that IPA uses to
>>>>> authenticate to dogtag. If it doesn't exist, or is expired,
>>>>> or is the wrong one, then authentication will fail.
>>>>> 
>>>>> The cert is shared amongst all the IPA masters, so if it
>>>>> is working on one master then fixing the replica should be 
>>>>> straightforward assuming it already has the key.
>>>>> 
>>>>> rob
>>>> 
>>>> getcert list -d /etc/httpd/alias -n ipaCert Number of 
>>>> certificates and requests being tracked: 9. Request ID 
>>>> '20130221171049': status: MONITORING stuck: no key pair
>>>> storage: 
>>>> type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
>>>>
>>>>
>>
>>>> 
Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
>>>> certificate: 
>>>> type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
>>>>
>>>>
>>
>>>> 
Certificate DB'
>>>> CA: dogtag-ipa-retrieve-agent-submit issuer: CN=Certificate 
>>>> Authority,O=ABAQIS.COM subject: CN=IPA RA,O=ABAQIS.COM
>>>> expires: 2013-12-10 03:23:26 UTC eku:
>>>> id-kp-serverAuth,id-kp-clientAuth pre-save command: post-save
>>>> command: /usr/lib64/ipa/certmonger/restart_httpd track: yes
>>>> auto-renew: yes
>>>> 
>>>> All the components appear to be there, the certificate is
>>>> valid until the 10th as you can see about. The other two
>>>> commands worked fine as well and everything appears to me to
>>>> be valid.
>>>> 
>>>> However, I am still getting the auth errors, and I note in
>>>> the log, what I assume to be the first ipa server attempting
>>>> to connect and getting auth errors as well.
>>> 
>>> Ok, I'm a little unclear on something. Can any of your IPA
>>> masters communicate with dogtag? I thought that one master
>>> could and one couldn't.
>>> 
>>> A simple way to find out is:
>>> 
>>> # ipa cert-show 1
>>> 
>>> This shows some random cert. If you get cert output then
>>> things are working. If not the logs will probably confirm the
>>> auth failure.
>>> 
>>> Next step is to run this on each master:
>>> 
>>> # certutil -L -d /etc/httpd/alias -n ipaCert | grep Serial
>>> 
>>> The serial numbers should be the same. I wonder if you'll find
>>> the to be different.
>>> 
>>> The serial number should match this value:
>>> 
>>> # ldapsearch -x -h localhost -p 7389 -b 
>>> uid=ipara,ou=People,o=ipaca description
>>> 
>>> The second integer in description is the serial number.
>>> 
>>> rob
>> 
>> Yep that is correct, on one instance I can communicate on the
>> other I can't. Let's call them primary and secondary, the primary
>> was the original install from whence all the others came.
>> 
>> ipa cert-show 1 on primary works, fails on secondary.
>> 
>> Running certutil -L -d /etc/httpd/alias -n ipaCert | grep Serial
>> on primary renders this:
>> 
>> Serial Number: 136 (0x88) Serial Number: 92 (0x5c)
>> 
>> Running against secondary gives only this: Serial Number: 92
>> (0x5c)
>> 
>> So it looks like that is probably the cause of all this mess,
>> though how the primary got two certs? I dunno.
>> 
>> Finally running the ldap query on both hosts confirms that 136 is
>> the serial for the cert in question, so it appears that this
>> somehow got messed up on the secondary.
>> 
>> Sooo, with all that known (and thank you for the help, I figured
>> it was something like this but reading the dogtag docs was taking
>> me a good long while to suss out the issue), how does one fix
>> it?
> 
> NSS uses a database to store its certificates so it's possible
> (and legal) to have multiple certs for the same private key. NSS
> picks the best one (e.g. the one that's valid).
> 
> So what you need to do on the primary is:
> 
> # certutil -L -d /etc/httpd/alias -n ipaCert -a > /tmp/ra.crt
> 
> Edit that file. there will be two certs in BEGIN/END blocks. If
> memory serves the last one is the one you want, so remove the
> first. You can double-check this after updating the file with:
> 
> # openssl x509 -text -in /tmp/ra.crt
> 
> It should have only the cert with the latest serial #.
> 
> Now add it to your cert database:
> 
> # certutil -A -n ipaCert -d /etc/httpd/alias -t u,u,u -a -i
> /tmp/ra.crt # service httpd restart
> 
> Things should work now.
> 
> rob
> 

Rob,
Thanks so much for the help. It was the first certificate but other
than that you were spot on, we can't all be perfect ;). That fixed the
issue and I am now able to do cert operations on both hosts.

Any idea how this situation could have come about?

- -Erinn
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.15 (GNU/Linux)

iQEcBAEBAgAGBQJSnX2dAAoJENetaK3v/E7P8rEIALrZScud0/26PR8XuRutdOCc
5HtplJCSlEIYRcj0Kov9v79YEu9BR94MK2DgrrPPp3owHLzmyaKvR/0HP5/GrnKC
DijRPckqEwrKICMmol3IqFZGPg2xwygI5voG6artPGfVi58lphXacFC+Yu3OQk64
jxrUHnrCVXWXqqerZsBurXfXAf4JYp0mem+IUzjNhKCKSYjje9Mnnc9rJHiRSY5q
xwjvz+3OcAKae/OS7hBI1exNwxRwdhnIM+fxz2waCbLMKVEGikrj2wcljMw3sLog
Vg0+i7FqSaAo81KekSw0u9VmbO7ouwLoYLspRZRBs3TY72Zjq6ArFnisQI9m4NE=
=9Cdi
-----END PGP SIGNATURE-----