<div dir="ltr">Can anyone help? At this point I'm stuck and I may have to consider alternatives :(<div class="gmail_extra"> <div class="gmail_quote">On 21 February 2017 at 09:37, Tiemen Ruiten <<a href="mailto:t.ruiten@rdmedia.com" target="_blank">t.ruiten@rdmedia.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Flo,<div> </div><div>Do you have any pointers?</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"> <div class="gmail_quote">On 20 February 2017 at 10:05, Tiemen Ruiten <<a href="mailto:t.ruiten@rdmedia.com" target="_blank">t.ruiten@rdmedia.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hello Flo,</div><div> </div>Thanks for your response. I ran that command and I seem to have a different problem (connectors are defined as you indicated):<div> </div><div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">[tiemen@copernicum ~]$ sudo getcert list -d /etc/dirsrv/slapd-IPA-RDMEDIA-COM/ [sudo] password for tiemen: Number of certificates and requests being tracked: 2. Request ID '20170217130857': status: CA_UNREACHABLE ca-error: Server at <a href="https://moscovium.ipa.rdmedia.com/ipa/xml" target="_blank">https://moscovium.ipa.rdmedia.com/ipa/xml</a> failed request, will retry: 4301 (RPC failed at server. Certificate operation cannot be completed: FAILURE (CA not found: 1ba8130c-56b8-4bd9-ae8a-8b0333d71b80)). stuck: no key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-IPA-RDMEDIA-COM',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-IPA-RDMEDIA-COM//pwdfile.txt' certificate: type=NSSDB,location='/etc/dirsrv/slapd-IPA-RDMEDIA-COM',nickname='Server-Cert' CA: IPA issuer: subject: expires: unknown pre-save command: post-save command: track: yes auto-renew: yes</blockquote></div><div> </div><div> </div><div> <div> </div><div> </div></div></div></div><div class="m_-3252368020138142412HOEnZb"><div class="m_-3252368020138142412h5"><div class="gmail_extra"> <div class="gmail_quote">On 20 February 2017 at 09:28, Florence Blanc-Renaud <<a href="mailto:flo@redhat.com" target="_blank">flo@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 02/17/2017 10:36 AM, Tiemen Ruiten wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> I went through that bugreport, particularly this section... OK, I think I found the error. On the logs I get something like this *before* the failing dirsrv restart: 2017-01-14T03:41:28Z DEBUG [27/44]: retrieving DS Certificate 2017-01-14T03:41:28Z DEBUG Loading Index file from '/var/lib/ipa/sysrestore/sysrestore.index' 2017-01-14T03:41:28Z DEBUG Starting external process 2017-01-14T03:41:28Z DEBUG args=/usr/bin/certutil -d /etc/dirsrv/slapd-EXAMPLE-COM/ -L -n <a href="http://EXAMPLE.COM" rel="noreferrer" target="_blank">EXAMPLE.COM</a> <<a href="http://EXAMPLE.COM" rel="noreferrer" target="_blank">http://EXAMPLE.COM</a>> IPA CA -a 2017-01-14T03:41:28Z DEBUG Process finished, return code=255 2017-01-14T03:41:28Z DEBUG stdout= 2017-01-14T03:41:28Z DEBUG stderr=certutil: Could not find cert: <a href="http://EXAMPLE.COM" rel="noreferrer" target="_blank">EXAMPLE.COM</a> <<a href="http://EXAMPLE.COM" rel="noreferrer" target="_blank">http://EXAMPLE.COM</a>> IPA CA : PR_FILE_NOT_FOUND_ERROR: File not found </blockquote> Hi, this error shows that the server certificate for the LDAP server is not present in the NSS database. I am pretty sure that if you run $ getcert list -d /etc/dirsrv/slapd-DOMAIN you will get an error like this one: status: CA_UNREACHABLE ca-error: Server at <a href="https://ipa.EXAMPLE.COM/ipa/xml" rel="noreferrer" target="_blank">https://ipa.EXAMPLE.COM/ipa/xml</a> failed request, will retry: 4301 (RPC failed at server. Certificate operation cannot be completed: Unable to communicate with CMS (503)). Make sure that the file /etc/pki/pki-tomcat/server.xml (on all the masters) defines the AJP connector like this: <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" address="localhost" /> and that the /etc/hosts file (on all the masters) properly defines localhost: 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 Then restart the PKI service on the masters: systemctl stop pki-tomcatd@pki-tomcat.service After this, you should be able to re-run ipa-replica-install without any problem. HTH, Flo. <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> So, when the process stopped, I run the command again: # /usr/bin/certutil -d /etc/dirsrv/slapd-EXAMPLE-COM/ -L -n <a href="http://EXAMPLE.COM" rel="noreferrer" target="_blank">EXAMPLE.COM</a> <<a href="http://EXAMPLE.COM" rel="noreferrer" target="_blank">http://EXAMPLE.COM</a>> IPA CA -a certutil: Could not find cert: <a href="http://EXAMPLE.COM" rel="noreferrer" target="_blank">EXAMPLE.COM</a> <<a href="http://EXAMPLE.COM" rel="noreferrer" target="_blank">http://EXAMPLE.COM</a>> : PR_FILE_NOT_FOUND_ERROR: File not found and thought "wait... something is missing there": # /usr/bin/certutil -d /etc/dirsrv/slapd-EXAMPLE-COM/ -L -n "<a href="http://EXAMPLE.COM" rel="noreferrer" target="_blank">EXAMPLE.COM</a> <<a href="http://EXAMPLE.COM" rel="noreferrer" target="_blank">http://EXAMPLE.COM</a>> IPA CA" -a -----BEGIN CERTIFICATE----- <strip> -----END CERTIFICATE----- So, could this be the problem? ...and indeed when I run [tiemen@copernicum ipapython]$ sudo /usr/bin/certutil -d /etc/dirsrv/slapd-IPA-RDMEDIA-COM/ -L -n <a href="http://IPA.RDMEDIA.COM" rel="noreferrer" target="_blank">IPA.RDMEDIA.COM</a> <<a href="http://IPA.RDMEDIA.COM" rel="noreferrer" target="_blank">http://IPA.RDMEDIA.COM</a>> IPA CA -a [sudo] password for tiemen: certutil: Could not find cert: <a href="http://IPA.RDMEDIA.COM" rel="noreferrer" target="_blank">IPA.RDMEDIA.COM</a> <<a href="http://IPA.RDMEDIA.COM" rel="noreferrer" target="_blank">http://IPA.RDMEDIA.COM</a>> : PR_FILE_NOT_FOUND_ERROR: File not found and when I run [tiemen@copernicum ipapython]$ sudo /usr/bin/certutil -d /etc/dirsrv/slapd-IPA-RDMEDIA-COM/ -L -n "<a href="http://IPA.RDMEDIA.COM" rel="noreferrer" target="_blank">IPA.RDMEDIA.COM</a> <<a href="http://IPA.RDMEDIA.COM" rel="noreferrer" target="_blank">http://IPA.RDMEDIA.COM</a>> IPA CA" -a -----BEGIN CERTIFICATE----- <snip> -----END CERTIFICATE----- valid certificate output. Where can I change this command to quote this string? On 16 February 2017 at 17:29, Jeff Goddard <<a href="mailto:jgoddard@emerlyn.com" target="_blank">jgoddard@emerlyn.com</a> <mailto:<a href="mailto:jgoddard@emerlyn.com" target="_blank">jgoddard@emerlyn.com</a>>> wrote: Might be another instance of this: <a href="https://fedorahosted.org/freeipa/ticket/6613" rel="noreferrer" target="_blank">https://fedorahosted.org/freeipa/ticket/6613</a> <<a href="https://fedorahosted.org/freeipa/ticket/6613" rel="noreferrer" target="_blank">https://fedorahosted.org/freeipa/ticket/6613</a>> Jeff On Thu, Feb 16, 2017 at 11:21 AM, Tiemen Ruiten <div><div class="m_-3252368020138142412m_-6868722775529461789h5"> <<a href="mailto:t.ruiten@rdmedia.com" target="_blank">t.ruiten@rdmedia.com</a> <mailto:<a href="mailto:t.ruiten@rdmedia.com" target="_blank">t.ruiten@rdmedia.com</a>>> wrote: Hello, I'm trying to add a third replica to a FreeIPA 4.4 domain (level 1), but I'm getting this error: [tiemen@copernicum ~]$ sudo ipa-replica-install -P admin -w "XXXXXXXXXX" --mkhomedir --setup-dns --forwarder 8.8.8.8 --forwarder 8.8.4.4 Checking DNS forwarders, please wait ... Run connection check to master Connection check OK Configuring NTP daemon (ntpd) [1/4]: stopping ntpd [2/4]: writing configuration [3/4]: configuring ntpd to start on boot [4/4]: starting ntpd Done configuring NTP daemon (ntpd). Configuring directory server (dirsrv). Estimated time: 1 minute [1/44]: creating directory server user [2/44]: creating directory server instance [3/44]: updating configuration in dse.ldif [4/44]: restarting directory server [5/44]: adding default schema [6/44]: enabling memberof plugin [7/44]: enabling winsync plugin [8/44]: configuring replication version plugin [9/44]: enabling IPA enrollment plugin [10/44]: enabling ldapi [11/44]: configuring uniqueness plugin [12/44]: configuring uuid plugin [13/44]: configuring modrdn plugin [14/44]: configuring DNS plugin [15/44]: enabling entryUSN plugin [16/44]: configuring lockout plugin [17/44]: configuring topology plugin [18/44]: creating indices [19/44]: enabling referential integrity plugin [20/44]: configuring certmap.conf [21/44]: configure autobind for root [22/44]: configure new location for managed entries [23/44]: configure dirsrv ccache [24/44]: enabling SASL mapping fallback [25/44]: restarting directory server [26/44]: creating DS keytab [27/44]: retrieving DS Certificate [28/44]: restarting directory server ipa : CRITICAL Failed to restart the directory server (Command '/bin/systemctl restart dirsrv@IPA-RDMEDIA-COM.service' returned non-zero exit status 1). See the installation log for details. [29/44]: setting up initial replication [error] error: [Errno 111] Connection refused Your system may be partly configured. Run /usr/sbin/ipa-server-install --uninstall to clean up. ipa.ipapython.install.cli.install_tool(Replica): ERROR [Errno 111] Connection refused ipa.ipapython.install.cli.install_tool(Replica): ERROR The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information In /var/log/ipareplica-install.log we find: 2017-02-16T15:53:59Z DEBUG [27/44]: retrieving DS Certificate 2017-02-16T15:53:59Z DEBUG Loading Index file from '/var/lib/ipa/sysrestore/sysrestore.index' 2017-02-16T15:53:59Z DEBUG Starting external process 2017-02-16T15:53:59Z DEBUG args=/usr/bin/certutil -d /etc/dirsrv/slapd-IPA-RDMEDIA-COM/ -L -n <a href="http://IPA.RDMEDIA.COM" rel="noreferrer" target="_blank">IPA.RDMEDIA.COM</a> </div></div> <<a href="http://IPA.RDMEDIA.COM" rel="noreferrer" target="_blank">http://IPA.RDMEDIA.COM</a>> IPA CA -a 2017-02-16T15:53:59Z DEBUG Process finished, return code=255 2017-02-16T15:53:59Z DEBUG stdout= *2017-02-16T15:53:59Z DEBUG stderr=certutil: Could not find cert: <a href="http://IPA.RDMEDIA.COM" rel="noreferrer" target="_blank">IPA.RDMEDIA.COM</a> <<a href="http://IPA.RDMEDIA.COM" rel="noreferrer" target="_blank">http://IPA.RDMEDIA.COM</a>> IPA CA : PR_FILE_NOT_FOUND_ERROR: File not found* 2017-02-16T15:53:59Z DEBUG Starting external process 2017-02-16T15:53:59Z DEBUG args=/usr/bin/certutil -d /etc/dirsrv/slapd-IPA-RDMEDIA-COM/ -N -f /etc/dirsrv/slapd-IPA-RDMEDIA-COM//pwdfile.txt 2017-02-16T15:53:59Z DEBUG Process finished, return code=0 2017-02-16T15:53:59Z DEBUG stdout= 2017-02-16T15:53:59Z DEBUG stderr= 2017-02-16T15:53:59Z DEBUG Starting external process 2017-02-16T15:53:59Z DEBUG args=/usr/bin/certutil -d /etc/dirsrv/slapd-IPA-RDMEDIA-COM/ -A -n <a href="http://IPA.RDMEDIA.COM" rel="noreferrer" target="_blank">IPA.RDMEDIA.COM</a> <<a href="http://IPA.RDMEDIA.COM" rel="noreferrer" target="_blank">http://IPA.RDMEDIA.COM</a>> IPA CA -t CT,C,C -a<div><div class="m_-3252368020138142412m_-6868722775529461789h5"> 2017-02-16T15:53:59Z DEBUG Process finished, return code=0 2017-02-16T15:53:59Z DEBUG stdout= 2017-02-16T15:53:59Z DEBUG stderr= 2017-02-16T15:53:59Z DEBUG certmonger request is in state dbus.String(u'NEWLY_ADDED_READING_KEYINFO', variant_level=1) 2017-02-16T15:54:04Z DEBUG certmonger request is in state dbus.String(u'CA_UNREACHABLE', variant_level=1) 2017-02-16T15:54:04Z DEBUG flushing ldapi://%2fvar%2frun%2fslapd-IPA-RDMEDIA-COM.socket from SchemaCache 2017-02-16T15:54:04Z DEBUG retrieving schema for SchemaCache url=ldapi://%2fvar%2frun%2fslapd-IPA-RDMEDIA-COM.socket conn=<ldap.ldapobject.SimpleLDAPObject instance at 0x74efd40> 2017-02-16T15:54:05Z DEBUG duration: 5 seconds 2017-02-16T15:54:05Z DEBUG [28/44]: restarting directory server 2017-02-16T15:54:05Z DEBUG Starting external process 2017-02-16T15:54:05Z DEBUG args=/bin/systemctl --system daemon-reload 2017-02-16T15:54:05Z DEBUG Process finished, return code=0 2017-02-16T15:54:05Z DEBUG stdout= 2017-02-16T15:54:05Z DEBUG stderr= 2017-02-16T15:54:05Z DEBUG Starting external process 2017-02-16T15:54:05Z DEBUG args=/bin/systemctl restart dirsrv@IPA-RDMEDIA-COM.service 2017-02-16T15:54:06Z DEBUG Process finished, return code=1 2017-02-16T15:54:06Z DEBUG stdout= 2017-02-16T15:54:06Z DEBUG stderr=Job for dirsrv@IPA-RDMEDIA-COM.service failed because the control process exited with error code. See "systemctl status dirsrv@IPA-RDMEDIA-COM.service" and "journalctl -xe" for details. 2017-02-16T15:54:06Z CRITICAL Failed to restart the directory server (Command '/bin/systemctl restart dirsrv@IPA-RDMEDIA-COM.service' returned non-zero exit status 1). See the installation log for details. 2017-02-16T15:54:06Z DEBUG duration: 1 seconds 2017-02-16T15:54:06Z DEBUG [29/44]: setting up initial replication 2017-02-16T15:54:16Z DEBUG Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 449, in start_creation run_step(full_msg, method) File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 439, in run_step method() File "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", line 405, in __setup_replica self.dm_password) File "/usr/lib/python2.7/site-packages/ipaserver/install/replication.py", line 118, in enable_replication_version_checking conn.do_simple_bind(bindpw=dirman_passwd) File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1665, in do_simple_bind self.__bind_with_wait(self.simple_bind, timeout, binddn, bindpw) File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1660, in __bind_with_wait self.__wait_for_connection(timeout) File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1643, in __wait_for_connection wait_for_open_socket(lurl.hostport, timeout) File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 1286, in wait_for_open_socket raise e error: [Errno 111] Connection refused 2017-02-16T15:54:16Z DEBUG [error] error: [Errno 111] Connection refused 2017-02-16T15:54:16Z DEBUG Destroyed connection context.ldap2_78478480 2017-02-16T15:54:16Z DEBUG File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171, in execute return_value = self.run() File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line 318, in run cfgr.run() File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 310, in run self.execute() File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 332, in execute for nothing in self._executor(): File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 372, in __runner self._handle_exception(exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, in _handle_exception six.reraise(*exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 362, in __runner step() File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 359, in <lambda> step = lambda: next(self.__gen) File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from six.reraise(*exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from value = gen.send(prev_value) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 586, in _configure next(executor) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 372, in __runner self._handle_exception(exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 449, in _handle_exception self.__parent._handle_exception(exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, in _handle_exception six.reraise(*exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 446, in _handle_exception super(ComponentBase, self)._handle_exception(exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 394, in _handle_exception six.reraise(*exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 362, in __runner step() File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 359, in <lambda> step = lambda: next(self.__gen) File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 81, in run_generator_with_yield_from six.reraise(*exc_info) File "/usr/lib/python2.7/site-packages/ipapython/install/util.py", line 59, in run_generator_with_yield_from value = gen.send(prev_value) File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line 63, in _install for nothing in self._installer(self.parent): File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 1714, in main promote(self) File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 364, in decorated func(installer) File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 1415, in promote promote=True, pkcs12_info=dirsrv_pkcs12_info) File "/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py", line 127, in install_replica_ds api=remote_api, File "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", line 399, in create_replica self.start_creation(runtime=60) File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 449, in start_creation run_step(full_msg, method) File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 439, in run_step method() File "/usr/lib/python2.7/site-packages/ipaserver/install/dsinstance.py", line 405, in __setup_replica self.dm_password) File "/usr/lib/python2.7/site-packages/ipaserver/install/replication.py", line 118, in enable_replication_version_checking conn.do_simple_bind(bindpw=dirman_passwd) File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1665, in do_simple_bind self.__bind_with_wait(self.simple_bind, timeout, binddn, bindpw) File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1660, in __bind_with_wait self.__wait_for_connection(timeout) File "/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1643, in __wait_for_connection wait_for_open_socket(lurl.hostport, timeout) File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 1286, in wait_for_open_socket raise e 2017-02-16T15:54:16Z DEBUG The ipa-replica-install command failed, exception: error: [Errno 111] Connection refused 2017-02-16T15:54:16Z ERROR [Errno 111] Connection refused 2017-02-16T15:54:16Z ERROR The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information How can I troubleshoot this? -- Tiemen Ruiten Systems Engineer R&D Media -- Manage your subscription for the Freeipa-users mailing list: <a href="https://www.redhat.com/mailman/listinfo/freeipa-users" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/freeipa-users</a> <<a href="https://www.redhat.com/mailman/listinfo/freeipa-users" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/freeipa-users</a>> Go to <a href="http://freeipa.org" rel="noreferrer" target="_blank">http://freeipa.org</a> for more info on the project -- Tiemen Ruiten Systems Engineer R&D Media </div></div></blockquote> </blockquote></div> <div> </div>-- <div class="m_-3252368020138142412m_-6868722775529461789gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">Tiemen Ruiten Systems Engineer R&D Media </div></div> </div> </div></div></blockquote></div> <div> </div>-- <div class="m_-3252368020138142412gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">Tiemen Ruiten Systems Engineer R&D Media </div></div> </div> </div></div></blockquote></div> <div> </div>-- <div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">Tiemen Ruiten Systems Engineer R&D Media </div></div> </div></div>