[Freeipa-users] ipa-replica-install failing, dirsrv not starting properly during install process

Florence Blanc-Renaud flo at redhat.com
Wed Nov 30 16:26:18 UTC 2016


On 11/30/2016 03:27 PM, David Dejaeghere wrote:
> Hi,
>
> The Pki service is running and I cannot find any issues with it.  I can
> run a curl request to the master hostname on port 8443 and communication
> works fine.
> Any other idea why this replica install code would fail and log
> CA_UNREACHABLE?
>
Hi,

can you check the logs on the server around the time of the replica 
installation?
- in /var/log/httpd/access_log you should find a line with
"POST https://ipaserver.domain.com:443/ca/eeca/ca/profileSubmitSSLClient 
HTTP/1.1" 200 2216

This line shows that certmonger did send the certificate request to IPA 
master.
- in /var/log/httpd/error_log, around the same time, you may find
[proxy:error] [pid 20702] (111)Connection refused: AH00957: AJP: attempt 
to connect to 127.0.0.1:8009 (localhost) failed
[proxy:error] [pid 20702] AH00959: ap_proxy_connect_backend disabling 
worker for (localhost) for 60s
[proxy_ajp:error] [pid 20702] [client <IPv6 address>] AH00896: failed to 
make connection to backend: localhost
[:error] [pid 20698] ipa: ERROR: ra.request_certificate(): Unable to 
communicate with CMS (503)
[:error] [pid 20698] ipa: INFO: [xmlserver] 
host/ipasrerver.domain.com at DOMAIN.COM: cert_request(u'[...]', 
principal=u'ldap/ipaclient.domain.com at DOMAIN.COM', add=True, 
version=u'2.51'): CertificateOperationError

If you find this type of error, the problem may come from the 
redirection httpd -> tomcat.
Apache is configured to redirect the URL profileSubmitSSLClient to 
ajp://localhost:8009 (see in /etc/httpd/conf.d/ipa-pki-proxy.conf).

You can check if Dogtag is listening on port 8009 (with netstat -tupnl | 
grep 8009, which should output the pid of Dogtag). If it is not the 
case, there is probably a configuration issue on Tomcat side.

Flo.

> Regards,
>
> David
>
>
> 2016-11-29 22:16 GMT+01:00 Florence Blanc-Renaud <flo at redhat.com
> <mailto:flo at redhat.com>>:
>
>     On 11/29/2016 03:19 PM, David Dejaeghere wrote:
>
>         Can you give me a couple of test commands?
>         I am not familiar with Dogtag.
>
>     Hi,
>
>     To reproduce the issue:
>     1. install IPA server
>     2. On the replica, run ipa-client-install
>     3. On the server, stop dogtag with
>     $ systemctl stop pki-tomcatd at pki-tomcat.service
>     4. On the replica, run ipa-replica-install
>
>     When you want to restart dogtag, you can run
>     $ systemctl start pki-tomcatd at pki-tomcat.service
>
>     If you want to check if dogtag is running:
>     $ systemctl status pki-tomcatd at pki-tomcat.service
>
>     You may find more information on Dogtag here:
>     http://pki.fedoraproject.org/wiki/PKI_Main_Page
>     <http://pki.fedoraproject.org/wiki/PKI_Main_Page>
>     http://pki.fedoraproject.org/wiki/IPA
>     <http://pki.fedoraproject.org/wiki/IPA>
>     http://pki.fedoraproject.org/wiki/Debugging_the_state_of_dogtag_in_an_ipa_install
>     <http://pki.fedoraproject.org/wiki/Debugging_the_state_of_dogtag_in_an_ipa_install>
>
>     Flo
>
>         Groeten,
>
>         David
>
>         2016-11-29 14:57 GMT+01:00 David Kupka <dkupka at redhat.com
>         <mailto:dkupka at redhat.com>
>         <mailto:dkupka at redhat.com <mailto:dkupka at redhat.com>>>:
>
>             On 29/11/16 13:55, David Dejaeghere wrote:
>
>                 Correct.  Same symptoms.
>
>                 2016-11-29T10:29:42Z DEBUG certmonger request is in state
>                 dbus.String(u'CA_UNREACHABLE', variant_level=1)
>
>                 Fedora 24 Server
>
>                 [root at ns02 ~]# dnf history userinstalled
>                 Packages installed by user
>                 freeipa-client-4.3.2-2.fc24.x86_64
>                 freeipa-server-4.3.2-2.fc24.x86_64
>                 grub2-1:2.02-0.34.fc24.x86_64
>                 kernel-4.5.5-300.fc24.x86_64
>                 kernel-4.8.8-200.fc24.x86_64
>                 lvm2-2.02.150-2.fc24.x86_64
>                 xfsprogs-4.5.0-2.fc24.x86_64
>
>
>             Ok. I've reproduced it by simply stopping dogtag on FreeIPA
>         server
>             while installing the replica. I see the exactly same errors as
>             you've reported and are described in the ticket, now.
>
>             Is dogtag running on your master? Is in responding (e.g. issuing
>             certificates for users)? Is it accessible from the replica?
>
>
>
>                 2016-11-29 13:41 GMT+01:00 Petr Vobornik
>         <pvoborni at redhat.com <mailto:pvoborni at redhat.com>
>                 <mailto:pvoborni at redhat.com <mailto:pvoborni at redhat.com>>>:
>
>
>                     On 11/29/2016 12:43 PM, David Kupka wrote:
>
>                         On 29/11/16 12:15, David Dejaeghere wrote:
>
>                             Seems like it is but it does not show a
>         server cert
>                             for dirsrv
>
>                             [root at ns02 ~]# ls -lZ
>         /etc/dirsrv/slapd-SOMETHING-BE/
>                             total 468
>                             -rw-------. 1 dirsrv root
>                              unconfined_u:object_r:dirsrv_config_t:s0
>                             65536
>                             Nov 29 11:29 cert8.db
>                             -rw-rw----. 1 dirsrv dirsrv
>                             unconfined_u:object_r:dirsrv_config_t:s0
>                             65536
>                             Nov 29 11:29 cert8.db.orig
>                             -r--r-----. 1 dirsrv dirsrv
>                             unconfined_u:object_r:dirsrv_config_t:s0
>                             1623
>                             Nov 29 11:29 certmap.conf
>                             -rw-------. 1 dirsrv dirsrv
>                             system_u:object_r:dirsrv_config_t:s0
>                             89977
>                             Nov 29 11:29 dse.ldif
>                             -rw-------. 2 dirsrv dirsrv
>                             system_u:object_r:dirsrv_config_t:s0
>                             89977
>                             Nov 29 11:29 dse.ldif.bak
>                             -rw-------. 2 dirsrv dirsrv
>                             system_u:object_r:dirsrv_config_t:s0
>                             89977
>                             Nov 29 11:29 dse.ldif.startOK
>                             -r--r-----. 1 dirsrv dirsrv
>                             unconfined_u:object_r:dirsrv_config_t:s0
>                             36228
>                             Nov 29 11:28 dse_original.ldif
>                             -rw-------. 1 dirsrv root
>                              unconfined_u:object_r:dirsrv_config_t:s0
>                             16384
>                             Nov 29 11:29 key3.db
>                             -rw-rw----. 1 dirsrv dirsrv
>                             unconfined_u:object_r:dirsrv_config_t:s0
>                             16384
>                             Nov 29 11:29 key3.db.orig
>                             -r--------. 1 dirsrv dirsrv
>                             unconfined_u:object_r:dirsrv_config_t:s0    66
>                             Nov 29 11:29 pin.txt
>                             -rw-------. 1 dirsrv dirsrv
>                             unconfined_u:object_r:dirsrv_config_t:s0    40
>                             Nov 29 11:29 pwdfile.txt
>                             drwxrwx---. 2 dirsrv dirsrv
>                             unconfined_u:object_r:dirsrv_config_t:s0
>                             4096
>                             Nov 29 11:29 schema
>                             -rw-------. 1 dirsrv root
>                              unconfined_u:object_r:dirsrv_config_t:s0
>                             16384
>                             Nov 29 11:29 secmod.db
>                             -rw-rw----. 1 dirsrv dirsrv
>                             unconfined_u:object_r:dirsrv_config_t:s0
>                             16384
>                             Nov 29 11:29 secmod.db.orig
>                             -r--r-----. 1 dirsrv dirsrv
>                             unconfined_u:object_r:dirsrv_config_t:s0
>                             15142
>                             Nov 29 11:28 slapd-collations.conf
>
>                             [root at ns02 ~]# certutil -d
>                             /etc/dirsrv/slapd-SOMETHING-BE -L
>
>                             Certificate Nickname
>                                      Trust
>                             Attributes
>
>                              SSL,S/MIME,JAR/XPI
>
>                             CN=something-PAPRIKA-CA,DC=something,DC=local
>                             CT,C,C
>                             SOMETHING.BE <http://SOMETHING.BE>
>         <http://SOMETHING.BE> IPA CA
>                                                          CT,C,C
>                             [root at ns02 ~]# certutil -d
>                             /etc/dirsrv/slapd-SOMETHING-BE -L
>
>                             Certificate Nickname
>                                      Trust
>                             Attributes
>
>                              SSL,S/MIME,JAR/XPI
>
>                             CN=something-PAPRIKA-CA,DC=something,DC=local
>                             CT,C,C
>                             SOMETHING.BE <http://SOMETHING.BE>
>         <http://SOMETHING.BE> IPA CA
>                                                          CT,C,C
>
>                             [root at ns02 ~]# ausearch -m avc -i
>                             <no matches>
>
>
>
>                         Exactly, the NSSDB should be accessible to
>         dirsrv and is
>                         missing the
>                         Server-Cert but I don't understand why there's "bad
>                         database" error in
>                         the errors log. I'll try to reproduce it. What
>         version
>                         of FreeIPA are
>                         you using? On what system?
>
>
>                     Right.
>
>                     Seems bit similar to
>                     https://fedorahosted.org/freeipa/ticket/6514
>         <https://fedorahosted.org/freeipa/ticket/6514>
>                     <https://fedorahosted.org/freeipa/ticket/6514
>         <https://fedorahosted.org/freeipa/ticket/6514>> would
>                     be good to check if it has the same symptoms, mainly
>                       certmonger request is in state
>         dbus.String(u'CA_UNREACHABLE',
>                     variant_level=1)
>
>                     in replica install log.
>
>
>
>
>                             2016-11-29 12:09 GMT+01:00 David Kupka
>                             <dkupka at redhat.com
>         <mailto:dkupka at redhat.com> <mailto:dkupka at redhat.com
>         <mailto:dkupka at redhat.com>>>:
>
>
>                                 On 29/11/16 11:51, David Dejaeghere wrote:
>
>                                     Hi,
>
>                                     I have a setup where i want to add a
>                                     replica.  The first master
>                                     setup has
>                                     an externally signed cert for dirsrv and
>                                     httpd.  The replica is
>                                     prepapred
>                                     succesfully with ipa-client-install
>         but the
>                                     replica install then keeps
>                                     failing.  It seems that during install
>                                     dirserv is not configured
>                                     correctly
>                                     with a valid server certificate.
>         Output from
>                                     the dirsrv error added to
>                                     this
>                                     email as well.
>
>                                     [root at ns02 ~]# ipa-replica-install
>         --setup-ca
>                                     WARNING: conflicting time&date
>                                     synchronization service 'chronyd' will
>                                     be disabled in favor of ntpd
>
>                                     Run connection check to master
>                                     Connection check OK
>                                     Configuring NTP daemon (ntpd)
>                                       [1/4]: stopping ntpd
>                                       [2/4]: writing configuration
>                                       [3/4]: configuring ntpd to start
>         on boot
>                                       [4/4]: starting ntpd
>                                     Done configuring NTP daemon (ntpd).
>                                     Configuring directory server (dirsrv).
>                                     Estimated time: 1 minute
>                                       [1/43]: creating directory server user
>                                       [2/43]: creating directory server
>         instance
>                                       [3/43]: restarting directory server
>                                       [4/43]: adding default schema
>                                       [5/43]: enabling memberof plugin
>                                       [6/43]: enabling winsync plugin
>                                       [7/43]: configuring replication
>         version plugin
>                                       [8/43]: enabling IPA enrollment plugin
>                                       [9/43]: enabling ldapi
>                                       [10/43]: configuring uniqueness plugin
>                                       [11/43]: configuring uuid plugin
>                                       [12/43]: configuring modrdn plugin
>                                       [13/43]: configuring DNS plugin
>                                       [14/43]: enabling entryUSN plugin
>                                       [15/43]: configuring lockout plugin
>                                       [16/43]: configuring topology plugin
>                                       [17/43]: creating indices
>                                       [18/43]: enabling referential
>         integrity plugin
>                                       [19/43]: configuring certmap.conf
>                                       [20/43]: configure autobind for root
>                                       [21/43]: configure new location for
>                                     managed entries
>                                       [22/43]: configure dirsrv ccache
>                                       [23/43]: enabling SASL mapping
>         fallback
>                                       [24/43]: restarting directory server
>                                       [25/43]: creating DS keytab
>                                       [26/43]: retrieving DS Certificate
>                                       [27/43]: restarting directory server
>                                     ipa         : CRITICAL Failed to
>         restart the
>                                     directory server (Command
>                                     '/bin/systemctl restart
>                                     dirsrv at SOMETHING-BE.service' returned
>
>                     non-zero
>
>                                     exit
>                                     status 1). See the installation log
>         for details.
>                                       [28/43]: setting up initial
>         replication
>                                       [error] error: [Errno 111]
>         Connection refused
>                                     Your system may be partly configured.
>                                     Run /usr/sbin/ipa-server-install
>         --uninstall
>                                     to clean up.
>
>
>                                     [29/Nov/2016:11:29:44.034285579
>         +0100] SSL
>                                     alert: Security
>                                     Initialization:
>                                     Can't find certificate (Server-Cert)
>         for family
>                                     cn=RSA,cn=encryption,cn=config (Netscape
>                                     Portable Runtime error -8174
>
>                     -
>
>                                     security library: bad database.)
>                                     [29/Nov/2016:11:29:44.045039728
>         +0100] SSL
>                                     alert: Security
>                                     Initialization:
>                                     Unable to retrieve private key for cert
>                                     Server-Cert of family
>                                     cn=RSA,cn=encryption,cn=config (Netscape
>                                     Portable Runtime error -8174
>
>                     -
>
>                                     security library: bad database.)
>
>
>
>
>                                 Hello David,
>
>                                 The error from the log indicates that
>         either the
>                                 NSSDB for dirsrv is
>
>                     not
>
>                                 initialized or not accessible.
>
>                                 Could you please send output of the
>         following
>                                 commands?
>
>                                 # ls -lZ /etc/dirsrv/slapd-$REALM/
>                                 # certutil -d /etc/dirsrv/slapd-$REALM/ -L
>                                 # ausearch -m avc -i
>
>
>                                 --
>                                 David Kupka
>
>
>
>                     --
>                     Petr Vobornik
>
>
>
>
>             --
>             David Kupka
>
>
>
>
>
>




More information about the Freeipa-users mailing list