<div dir="ltr"><div><div><div>Well Flo, the issue was IPV6 was disabled. As soon as I enabled it again, added that /etc/hosts entry back, and rebooted the server, things are working again! </div>So is that now a requirement for 4.4.x? Server worked fine on 4.2 without IPV6 enabled. Or has that always been a requirement and I just got lucky somehow. I'm looking through the docs now. Regardless, thank you very much for the help Flo! </div> </div>Jay </div><div class="gmail_extra"> <div class="gmail_quote">On Tue, Dec 13, 2016 at 10:20 AM, Florence Blanc-Renaud <<a href="mailto:flo@redhat.com" target="_blank">flo@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 12/13/2016 04:41 PM, jay wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Maybe this will help more, I noticed this error in the Apache logs [Tue Dec 13 09:33:37.<a href="tel:774921%202016" value="+17749212016" target="_blank">774921 2016</a>] [:error] [pid 2309] ipa: INFO: [jsonserver_kerb] <a href="mailto:admin@IPA.US-WEST-2.COMPUTE.IN">admin@IPA.US-WEST-2.COMPUTE.IN</a>TERNAL: cert_show/1(u'1', version=u'2.213'): CertificateOperationError [Tue Dec 13 09:35:29.141847 2016] [proxy:error] [pid 2316] (111)Connection refused: AH00957: AJP: attempt to connect to <a href="http://127.0.0.1:8009" rel="noreferrer" target="_blank">127.0.0.1:8009</a> <<a href="http://127.0.0.1:8009" rel="noreferrer" target="_blank">http://127.0.0.1:8009</a>> (localhost) failed [Tue Dec 13 09:35:29.141881 2016] [proxy:error] [pid 2316] AH00959: ap_proxy_connect_backend disabling worker for (localhost) for 60s [Tue Dec 13 09:35:29.141900 2016] [proxy_ajp:error] [pid 2316] [client <a href="http://172.31.0.254:39646" rel="noreferrer" target="_blank">172.31.0.254:39646</a> <<a href="http://172.31.0.254:39646" rel="noreferrer" target="_blank">http://172.31.0.254:39646</a>>] AH00896: failed to make connection to backend: localhost [Tue Dec 13 09:35:29.142412 2016] [:error] [pid 2310] ipa: ERROR: ra.get_certificate(): Unable to communicate with CMS (503) So whatever is running on port 8009 isn't responding or setup properly. Jay On Tue, Dec 13, 2016 at 8:46 AM, jay <<a href="mailto:titleistfour@gmail.com" target="_blank">titleistfour@gmail.com</a> <div><div class="h5"> <mailto:<a href="mailto:titleistfour@gmail.com" target="_blank">titleistfour@gmail.com</a>>> wrote: Thank you for the response Flo. So I do see Apache running and listening on port 443. However, running that command I get a 503 * Trying 172.31.0.254... * Connected to ip-172-31-0-254.us-west-2.compute.internal (172.31.0.254) port 443 (#0) * Initializing NSS with certpath: sql:/etc/httpd/alias * CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none * SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 * Server certificate: * subject: CN=ip-172-31-0-254.us-west-2.compute.internal,O=IPA.US-WEST-2.COMPUTE.INTERNAL * start date: Dec 13 14:33:16 2016 GMT * expire date: Dec 14 14:33:16 2018 GMT * common name: ip-172-31-0-254.us-west-2.compute.internal * issuer: CN=Certificate Authority,O=IPA.US-WEST-2.COMPUTE.INTERNAL > GET /ca/agent/ca/displayBySerial?serialNumber=1 HTTP/1.1 > User-Agent: curl/7.29.0 > Host: ip-172-31-0-254.us-west-2.compute.internal > Accept: */* > * NSS: using client certificate: ipaCert * subject: CN=IPA RA,O=<a href="http://IPA.US-WEST-2.COMPUTE.INT">IPA.US-WEST-2.COMPUTE.INT</a>ERNAL * start date: Dec 13 14:32:28 2016 GMT * expire date: Dec 03 14:32:28 2018 GMT * common name: IPA RA * issuer: CN=Certificate Authority,O=IPA.US-WEST-2.COMPUTE.INTERNAL < HTTP/1.1 503 Service Unavailable < Date: Tue, 13 Dec 2016 14:44:00 GMT < Server: Apache < Content-Length: 299 < Connection: close < Content-Type: text/html; charset=iso-8859-1 [root@ip-172-31-0-254 ~]# cat out.html <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>503 Service Unavailable</title> </head><body> <h1>Service Unavailable</h1> The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later. </body></html> [root@ip-172-31-0-254 ~]# What would cause the service to be unavailable? Maybe the installer changed and I need to provide another option now that I didn't have to before the version upgrade? </div></div></blockquote> Hi Jay, I am not completely familiar with Tomcat, but I understand so far that the httpd server is configured to redirect requests on displayBySerial to Tomcat with this setting in the file /etc/httpd/conf.d/ipa-pki-proxy.conf: <LocationMatch "^/ca/agent/ca/displayBySerial|^/ca/agent/ca/doRevoke|^/ca/agent/ca/doUnrevoke|^/ca/agent/ca/updateDomainXML|^/ca/eeca/ca/profileSubmitSSLClient|^/kra/agent/kra/connector"> NSSOptions +StdEnvVars +ExportCertData +StrictRequire +OptRenegotiate NSSVerifyClient require ProxyPassMatch ajp://localhost:8009 ProxyPassReverse ajp://localhost:8009 </LocationMatch> And this port must be configured in /etc/pki/pki-tomcat/server.xml: <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" address="::1" /> In my setup I also have /etc/hosts defining localhost: 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 Can you check that you also have those settings? If yes, then I assume that Tomcat is not able to create the AJP connector on port 8009. When PKI is started, you should find a trace of the connector initialization in /var/log/pki/pki-tomcat/catalina.$DATE.log: 12-Dec-2016 13:54:17.579 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["ajp-nio-0:0:0:0:0:0:0:1-8009"] 12-Dec-2016 13:54:25.076 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-0:0:0:0:0:0:0:1-8009"] 12-Dec-2016 13:56:33.683 INFO [ajp-nio-0:0:0:0:0:0:0:1-8009-exec-1] org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.processApplication RESTEASY002225: Deploying javax.ws.rs.core.Application: class org.dogtagpki.server.ca.rest.CAApplication Next steps to debug would be to increase Tomcat logs. Flo. <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> Thanks, Jay On Tue, Dec 13, 2016 at 1:56 AM, Florence Blanc-Renaud <div><div class="h5"> <<a href="mailto:flo@redhat.com" target="_blank">flo@redhat.com</a> <mailto:<a href="mailto:flo@redhat.com" target="_blank">flo@redhat.com</a>>> wrote: On 12/12/2016 10:32 PM, jay wrote: Hello, I have been testing freeipa on CentOS 7 for a while now with a relatively simple setup, just a single server and 12 or so Linux clients in AWS. I went to rebuild the environment today and part of my Ansible playbook failed with this error ipa: ERROR: Certificate operation cannot be completed: Unable to communicate with CMS (503) This is the command that failed /usr/bin/ipa cert-show 1 --out=/root/cacert.crt I noticed the version I was using on Friday was ipa-server-4.2.0-15.0.1.el7.centos.19.x86_64. But now I'm getting ipa-server-4.4.0-14.el7.centos.x86_64 installed, so the repo was updated over the weekend. Is there a known issue running cert-show with this version? I can't find anything in the debug logs that point to something wrong. Running 'ipa cert-find' and 'getcert list -d /etc/httpd/alias -n ipaCert' work just fine. Can someone offer some advice or pointer to what might be going on? I'm invoking the install with these options and it has worked flawlessly before this new version 2016-12-12T21:05:21Z DEBUG ipa-server-install was invoked with arguments [] and options: {'no_dns_ sshfp': None, 'ignore_topology_disconnect': None, 'verbose': False, 'ip_addresses': [CheckedIPAddr ess('172.31.0.235')], 'domainlevel': None, 'mkhomedir': None, 'http_cert_files': None, 'no_ntp': N one, 'reverse_zones': None, 'no_forwarders': None, 'external_ca_type': None, 'ssh_trust_dns': True , 'domain_name': 'ipa.us-west-2.compute.internal', 'idmax': None, 'http_cert_name': None, 'dirsrv_ cert_files': None, 'no_dnssec_validation': None, 'ca_signing_algorithm': None, 'no_reverse': None, 'subject': None, 'unattended': True, 'auto_reverse': None, 'auto_forwarders': None, 'no_host_dns' : None, 'no_sshd': None, 'no_ui_redirect': None, 'ignore_last_of_role': None, 'realm_name': 'IPA.U S-WEST-2.COMPUTE.INTERNAL', 'forwarders': [CheckedIPAddress('172.31.0.2')], 'idstart': 5000, 'exte rnal_ca': None, 'no_ssh': None, 'external_cert_files': None, 'no_hbac_allow': None, 'forward_polic y': None, 'dirsrv_cert_name': None, 'ca_cert_files': None, 'zonemgr': None, 'quiet': False, 'setup _dns': True, 'host_name': '<a href="http://ip-172-31-0-235.us-west-2.com" rel="noreferrer" target="_blank">ip-172-31-0-235.us-west-2.com</a> </div></div> <<a href="http://ip-172-31-0-235.us-west-2.com" rel="noreferrer" target="_blank">http://ip-172-31-0-235.us-west-2.com</a>>pute.internal', 'dirsrv_config_file': None , 'log_file': None, 'allow_zone_overlap': None, 'uninstall': False} 2016-12-12T21:05:21Z DEBUG IPA version 4.4.0-14.el7.centos Thank you Jay Hi, the ipa cert-show command is communicating with Dogtag, using port 443. Can you check if Dogtag is properly responding on this port? $ SSL_DIR=/etc/httpd/alias/ curl -v -E ipaCert:`cat /etc/httpd/alias/pwdfile.txt` <a href="https://hostname.domainname:443/ca/agent/ca/displayBySerial?serialNumber=1" rel="noreferrer" target="_blank">https://hostname.domainname:443/ca/agent/ca/displayBySerial?serialNumber=1</a> <<a href="https://hostname.domainname:443/ca/agent/ca/displayBySerial?serialNumber=1" rel="noreferrer" target="_blank">https://hostname.domainname:443/ca/agent/ca/displayBySerial?serialNumber=1</a>> -o out.html The issue can be that Dogtag is down, or a SSL issue (the certificate ipaCert in /etc/httpd/alias is used to authenticate the client to Dogtag). HTH, Flo. </blockquote> </blockquote></div> </div>