[Freeipa-users] ipa-getcert list fails to report correctly
Les Stott
Less at imagine-sw.com
Mon Feb 23 09:01:29 UTC 2015
> -----Original Message-----
> From: freeipa-users-bounces at redhat.com [mailto:freeipa-users-
> bounces at redhat.com] On Behalf Of Les Stott
> Sent: Monday, 23 February 2015 12:18 PM
> To: Rob Crittenden; Martin Kosek; freeipa-users at redhat.com; Endi Dewata;
> Jan Cholasta
> Subject: Re: [Freeipa-users] ipa-getcert list fails to report correctly
>
>
>
> > -----Original Message-----
> > From: Rob Crittenden [mailto:rcritten at redhat.com]
> > Sent: Saturday, 21 February 2015 1:39 AM
> > To: Martin Kosek; Les Stott; freeipa-users at redhat.com; Endi Dewata;
> > Jan Cholasta
> > Subject: Re: [Freeipa-users] ipa-getcert list fails to report
> > correctly
> >
> > Martin Kosek wrote:
> > > On 02/20/2015 06:56 AM, Les Stott wrote:
> > >> Hi all,
> > >>
> > >> The following is blocking the ability for me to install a CA replica.
> > >>
> > >> Environment:
> > >>
> > >> RHEL 6.6
> > >>
> > >> IPA 3.0.0-42
> > >>
> > >> PKI 9.0.3-38
> > >>
> > >> On the master the following is happening:
> > >>
> > >> ipa-getcert list
> > >>
> > >> Number of certificates and requests being tracked: 5.
> > >>
> > >> (but it shows no certificate details in the output)
> > >>
> > >> Running "getcert list" shows complete output.
> > >>
> > >> Also, when trying to browse
> > >> https://master.mydomain.com/ca/ee/ca/getCertChain i get a failed
> > >> response. The apache error logs on the master show....
> > >>
> > >> [Thu Feb 19 23:23:23 2015] [error] SSL Library Error: -12271 SSL
> > >> client cannot verify your certificate
> > >>
> > >> The reason I am trying to browse that address is because that's
> > >> what the ipa-ca-install setup is failing at (it complains that the
> > >> CA certificate is not in proper format, in fact it's not able to
> > >> get it at all).
> > >>
> > >> I know from another working ipa setup that ....
> > >>
> > >> Browsing to the above address provides valid xml content and
> > >> ipa-getcert list shows certificate details and not just the number
> > >> of tracked certificates.
> > >>
> > >> Been trying for a long time to figure out the issues without luck.
> > >>
> > >> I would greatly appreciate any help to troubleshoot and resolve the
> > >> above issues.
> > >>
> > >> Regards,
> > >>
> > >> Les
> > >
> > > Endi or JanC, would you have any advise for Les? To me, it looks
> > > like the Apache does not have proper certificate installed.
> > >
> > > My ipa-getcert on RHEL-6.6 shows 3 Server-Certs tracked, making it
> > > in total of 8 certs tracked:
> > >
> > > # ipa-getcert list
> > > Number of certificates and requests being tracked: 8.
> > > Request ID '20141111000002':
> > > status: MONITORING
> > > stuck: no
> > > key pair storage:
> > > type=NSSDB,location='/etc/dirsrv/slapd-IDM-LAB-BOS-REDHAT-
> > COM',nicknam
> > > e='Server-Cert',token='NSS
> > > Certificate
> > > DB',pinfile='/etc/dirsrv/slapd-IDM-LAB-BOS-REDHAT-COM/pwdfile.txt'
> > > certificate:
> > > type=NSSDB,location='/etc/dirsrv/slapd-IDM-LAB-BOS-REDHAT-
> > COM',nicknam
> > > e='Server-Cert',token='NSS
> > > Certificate DB'
> > > CA: IPA
> > > issuer: CN=Certificate Authority,O=EXAMPLE.COM
> > > subject: CN=vm-086.example.com,O=EXAMPLE.COM
> > > expires: 2016-11-11 00:00:01 UTC
> > > key usage:
> > > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> > > eku: id-kp-serverAuth,id-kp-clientAuth
> > > pre-save command:
> > > post-save command:
> > > track: yes
> > > auto-renew: yes
> > > Request ID '20141111000047':
> > > status: MONITORING
> > > stuck: no
> > > key pair storage:
> > > type=NSSDB,location='/etc/dirsrv/slapd-PKI-IPA',nickname='Server-Cert'
> > > ,token='NSS Certificate
> > > DB',pinfile='/etc/dirsrv/slapd-PKI-IPA/pwdfile.txt'
> > > certificate:
> > > type=NSSDB,location='/etc/dirsrv/slapd-PKI-IPA',nickname='Server-Cert'
> > > ,token='NSS
> > > Certificate DB'
> > > CA: IPA
> > > issuer: CN=Certificate Authority,O=EXAMPLE.COM
> > > subject: CN=vm-086.example.com,O=EXAMPLE.COM
> > > expires: 2016-11-11 00:00:46 UTC
> > > key usage:
> > > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> > > eku: id-kp-serverAuth,id-kp-clientAuth
> > > pre-save command:
> > > post-save command:
> > > track: yes
> > > auto-renew: yes
> > > Request ID '20141111000302':
> > > status: MONITORING
> > > stuck: no
> > > key pair storage:
> > > type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token=
> > > 'N SS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
> > > certificate:
> > > type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token=
> > > 'N
> > > SS
> > > Certificate DB'
> > > CA: IPA
> > > issuer: CN=Certificate Authority,O=EXAMPLE.COM
> > > subject: CN=vm-086.example.com,O=EXAMPLE.COM
> > > expires: 2016-11-11 00:03:02 UTC
> > > key usage:
> > > digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> > > eku: id-kp-serverAuth,id-kp-clientAuth
> > > pre-save command:
> > > post-save command:
> > > track: yes
> > > auto-renew: yes
> > >
> > >
> > > What is actually in your Apache NSS database?
> > >
> > > # certutil -L -d /etc/httpd/alias/
> > >
> > > Martin
> > >
> >
> > Remember ipa-getcert is just a shortcut for certificates using the
> > certmonger CA named IPA, so it's more a filter than anything else. I
> > don't know why it wouldn't display any output but I'd file a bug.
> >
> > I think we'd need to see the getcert list output to try to figure out
> > what is going on.
> >
> > As for the SSL error fetching the cert chain I think Martin may be
> > onto something. The request is proxied through Apache. I think the
> > client here might be the Apache proxy client.
> >
> > I believe this command replicates what Apache is doing, you might give
> > it a try on the master. This will get the chain directly from dogtag,
> > bypassing
> > Apache:
> >
> > $ curl -v --cacert /etc/ipa/ca.crt
> > https://`hostname`:9444/ca/ee/ca/getCertChain
> >
> > rob
>
> Certutil shows....
>
> certutil -L -d /etc/httpd/alias/
>
> Certificate Nickname Trust Attributes
> SSL,S/MIME,JAR/XPI
>
> MYDOMAIN.COM IPA CA CT,C,C
> ipaCert u,u,u
> Signing-Cert u,u,u
> Server-Cert u,u,u
>
> curl -v --cacert /etc/ipa/ca.crt
> https://`hostname`:9444/ca/ee/ca/getCertChain
> * About to connect() to `hostname` port 9444 (#0)
> * Trying 192.168.1.1... connected
> * Connected to `hostname` (192.168.1.1) port 9444 (#0)
> * Initializing NSS with certpath: sql:/etc/pki/nssdb
> * CAfile: /etc/ipa/ca.crt
> CApath: none
> * SSL connection using TLS_RSA_WITH_AES_128_CBC_SHA
> * Server certificate:
> * subject: CN=`hostname`,O=MYDOMAIN.COM
> * start date: Dec 13 01:21:30 2013 GMT
> * expire date: Dec 03 01:21:30 2015 GMT
> * common name: `hostname`
> * issuer: CN=Certificate Authority,O=MYDOMAIN.COM
> > GET /ca/ee/ca/getCertChain HTTP/1.1
> > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7
> > NSS/3.16.2.3 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> > Host: `hostname`:9444
> > Accept: */*
> >
> < HTTP/1.1 200 OK
> < Server: Apache-Coyote/1.1
> < Content-Type: application/xml
> < Content-Length: 1434
> < Date: Mon, 23 Feb 2015 01:04:29 GMT
> <
> <?xml version="1.0" encoding="UTF-8"
> standalone="no"?><XMLResponse><Status>0</Status><ChainBase64>MIID
> zwYJKoZIhvcNAQcCoIIDwDCCA7wCAQExADAPBgkqhkiG9w0BBwGgAgQAoII
> DoDCCA5wwggKEoAMCAQICAQEwDQYJKoZIhvcNAQELBQAwOjEYMBYGA1U
> EChMPREVSSVZBVElWRVMuQ09NMR4wHAYDVQQDExVDZXJ0aWZpY2F0ZSB
> BdXRob3JpdHkwHhcNMTMxMjEzMDEyMTI5WhcNMzMxMjEzMDEyMTI5Wj
> A6MRgwFgYDVQQKEw9ERVJJVkFUSVZFUy5DT00xHjAcBgNVBAMTFUNlcnRp
> ZmljYXRlIEF1dGhvcml0eTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCg
> gEBAMAA8EaYhmpjSA8o3/1kB/W1+0K6+FrwCS+njOgRtXhiTdmtSddXSDVxH
> OafFwqN26BR+QRPZbbpJY70gP3SG8W+J6+c37PMVNshWz6UfChGt6ubgFxlS
> TGUUre2Osr9I4C836MXpGJvRx2VDEuMUxv8j7B9iDRnTDglseqPqrMct2No4w
> k4cLtA9puBJb0Es76SOHP9edXlf6GBnuYwR8YMc1yJLqpP8IGpHhEkVxMsRpqk
> EpuuRwEFa7uBcTDhqVV24BpFlseZVubpiOdEgfb3IRBTjvI1Mum9OCJbuj9P/W
> mqMnrA0sQsmF/R3WBwFdMAsN3+bQCRw73+rwoeDNcCAwEAAaOBrDCBq
> TAfBgNVHSMEGDAWgBSO8J+j2jAuyg3a0yE+3oVCQJCWUTAPBgNVHRMBAf8
> EBTADAQH/MA4GA1UdDwEB/wQEAwIBxjAdBgNVHQ4EFgQUjvCfo9owLsoN
> 2tMhPt6FQkCQllEwRgYIKwYBBQUHAQEEOjA4MDYGCCsGAQUFBzABhipodHR
> wOi8vc2I!
> ybW9uMDEuZGVyaXZhdGl2ZXMuY29tOjgwL2* Connection #0 to host
> `hostname` left intact
> * Closing connection #0
> NhL29jc3AwDQYJKoZIhvcNAQELBQADggEBAKH8YkoTAzX2xNYMkZSDK84EK3
> e4FUixdXxc/EC5ehjrtaqXT1KT9Fl9DAF5/jYNKqgmEmtHnPGlfQ7/Y1ESdhEGcB
> ZjU4qLe4HaFXuw5c9odDYxhtjQUd1g7ifY8SKOcHDCY+6Xx6F/rhFgzrXXMndn8
> ZaYryctPoOAj/5INnLrJq8S4XyLmb2BHM4e1ORQbOhDi8xjhfK2veYXvIu55Brhp
> RSS/goz5oSE8e+QE/H9afRmeV2+WkS/YDhSyoUDb7CYjklRuONzX3GopKtp1y
> yLXQZnBFjCvIJvja0mo3ik3AXxSZuOwUIlV23U8CyPU/rDeiV00iUyA/fLvdkEtZkx
> AA==</ChainBase64></XMLResponse>
>
>
> In any event, I've decided to rebuilt my DR IPA environment. Late last year
> the master in DR had to be rebuilt due to a disk issue. While IPA was restored
> manually and appeared to be working fine, CA replication hasn't worked. I
> finally got CA replication working in Prod after enabling needed apache
> modules and performing a yum update to update related packages, but
> these things didn't help in DR. It's my strong suspicion that something got
> missed when restoring the DR master IPA server and this is what is causing all
> my grief. Therefore, I'm going to wipe it out and start from scratch in DR.
> There are other benefits for me to do this anyway.
>
Well things have gone from bad to worse.
I removed IPA in DR. uninstalled all ipa clients, uninstalled replicas, removed replication agreements and removed the master. Ran pki-remove to clear any leftover pki instances and used certutil -D to remove left behind ipa entries in /etc/httpd/alias.
So, clean slate to start again.
This time, in order to mirror config with prod, I began an installation for the master on a different server, let's call it serverb. It was previously a replica (in my prod environment, serverb is the true master, servera, serverc, and serverd are replicas).
So, trying to install a new fresh instance of IPA and it still fails to configure a CA.
Attached is the relevant portion of the server install log file (ipa-server-install.txt). I have removed certificate and copyright info to reduce its size. Also my server to install is serverb.mydomain.com
Apache logs at the time of the error show:
[Mon Feb 23 03:05:31 2015] [error] SSL Library Error: -12195 Peer does not recognize and trust the CA that issued your certificate
Certificate databases only show the following (note that "Server-Cert cert-pki-ca" got installed before the installer crashed). Prior to trying installation I had to manually remove server certs left behind from the previous installation via ...
certutil -d /etc/httpd/alias -D -n "Server-Cert"
certutil -d /etc/httpd/alias -D -n "MYDOMAIN.COM IPA CA"
certutil -d /etc/httpd/alias -D -n ipaCert
certutil -L -d /var/lib/pki-ca/alias
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
Server-Cert cert-pki-ca CTu,Cu,Cu
certutil -L -d /etc/pki/nssdb
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
Selinux is in permissive mode.
Ausearch -m avc does show some selinux issues, but its permissive mode so it should be ok right? In any event I have previously tried installing a CA replica with selinux disabled and it didn't help.
I have tried removing ipa and pki rpms and reinstalling. Then rerunning the ipa server install script but the same error occurs.
I noticed that /etc/ipa/ca.crt was still old, and referencing the original master. I removed that and again reran the installer but the same error occurred.
Note also that /etc/ipa/cr.crt was not recreated when ipa-python was reinstalled.
Other logs:
/var/log/pki-ca/system shows
5042.main - [23/Feb/2015:03:05:12 EST] [3] [3] Cannot build CA chain. Error java.security.cert.CertificateException: Certificate is not a PKCS #11 certificate
5042.main - [23/Feb/2015:03:05:12 EST] [13] [3] authz instance DirAclAuthz initialization failed and skipped, error=Property internaldb.ldapconn.port missing value
5042.http-9445-1 - [23/Feb/2015:03:05:26 EST] [3] [3] Cannot build CA chain. Error java.security.cert.CertificateException: Certificate is not a PKCS #11 certificate
5042.http-9445-1 - [23/Feb/2015:03:05:35 EST] [3] [3] CASigningUnit: Object certificate not found. Error org.mozilla.jss.crypto.ObjectNotFoundException
/var/log/pki-ca/catalina.out
Feb 23, 2015 3:05:11 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory ca
64-bit osutil library loaded
64-bit osutil library loaded
CMS Warning: FAILURE: Cannot build CA chain. Error java.security.cert.CertificateException: Certificate is not a PKCS #11 certificate|FAILURE: authz instance DirAclAuthz initialization failed and skipped, error=Property internaldb.ldapconn.port missing value|
Server is started.
Feb 23, 2015 3:05:12 AM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-9180
Feb 23, 2015 3:05:12 AM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-9443
Feb 23, 2015 3:05:12 AM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-9445
Feb 23, 2015 3:05:12 AM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-9444
Feb 23, 2015 3:05:12 AM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-9446
Feb 23, 2015 3:05:12 AM org.apache.jk.common.ChannelSocket init
INFO: JK: ajp13 listening on /0.0.0.0:9447
Feb 23, 2015 3:05:12 AM org.apache.jk.server.JkMain start
INFO: Jk running ID=0 time=0/25 config=null
Feb 23, 2015 3:05:12 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 1655 ms
I have no idea where to look next. There must be some remnant of the old system hanging around screwing things up but I cannot figure it out. This will drive me insane!
I can provide more logs if needed.
Thanks in advance for any help.
Regards,
Les
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ipa-server-install.txt
URL: <http://listman.redhat.com/archives/freeipa-users/attachments/20150223/2e87a9fd/attachment.txt>
More information about the Freeipa-users
mailing list