[Freeipa-devel] [PATCH] 762 Let the framework be able to override the hostname
Martin Kosek
mkosek at redhat.com
Mon May 30 12:05:48 UTC 2011
On Fri, 2011-05-27 at 15:39 -0400, Rob Crittenden wrote:
> Martin Kosek wrote:
> > On Wed, 2011-05-25 at 11:29 -0400, Rob Crittenden wrote:
> >> Martin Kosek wrote:
> >>> On Fri, 2011-04-01 at 11:47 -0400, Rob Crittenden wrote:
> >>>> The hostname is passed in during the server installation. We should use
> >>>> this hostname for the resulting server as well. It was being discarded
> >>>> and we always used the system hostname value.
> >>>>
> >>>> ticket 1052
> >>>>
> >>>> rob
> >>>
> >>> I have to NACK this again. I have a problem communicating with IPA on a
> >>> master machine. I reproduced in on 2 different machines. Please, correct
> >>> my steps if I am wrong, I do the following procedure
> >>>
> >>> 1) I prepare a fresh minimal F-15
> >>> 2) Install freeipa-server (current master with your patches)
> >>> 3) Add custom hostname to /etc/hosts
> >>> 4) Install IPA server:
> >>> ipa-server-install -p secret123 -a secret123 --hostname ipa.idm.lab.bos.redhat.com --setup-dns --forwarder=10.16.255.2
> >>> 5) # kinit admin
> >>> Password for admin at IDM.LAB.BOS.REDHAT.COM:
> >>> 6) # ipa user-show admin
> >>> ipa: ERROR: cannot connect to 'any of the configured servers':
> >>> https://ipa.idm.lab.bos.redhat.com/ipa/xml,
> >>> https://ipa.idm.lab.bos.redhat.com/ipa/xml
> >>>
> >>> # ping -c 1 ipa.idm.lab.bos.redhat.com
> >>> PING ipa.idm.lab.bos.redhat.com (10.16.78.140) 56(84) bytes of data.
> >>> 64 bytes from ipa.idm.lab.bos.redhat.com (10.16.78.140): icmp_req=1
> >>> ttl=64 time=0.049 ms
> >>>
> >>> Apache error_log shows relevant errors:
> >>>
> >>> [Wed May 25 06:42:38 2011] [error] ipa: ERROR: Failed to start IPA: Unable to retrieve LDAP schema: Invalid credentials: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Permission denied)
> >>> [Wed May 25 06:42:38 2011] [error] ipa: ERROR: Failed to start IPA: Unable to retrieve LDAP schema: Invalid credentials: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Permission denied)
> >>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
> >>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
> >>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
> >>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
> >>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
> >>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
> >>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
> >>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
> >>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
> >>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
> >>> [Wed May 25 06:43:56 2011] [notice] caught SIGTERM, shutting down
> >>> [Wed May 25 06:43:56 2011] [notice] SELinux policy enabled; httpd running as context system_u:system_r:kernel_t:s0
> >>> [Wed May 25 06:43:57 2011] [notice] Digest: generating secret for digest authentication ...
> >>> [Wed May 25 06:43:57 2011] [notice] Digest: done
> >>> [Wed May 25 06:43:57 2011] [notice] Apache/2.2.17 (Unix) DAV/2 mod_auth_kerb/5.4 mod_nss/2.2.17 NSS/3.12.9.0 mod_wsgi/3.2 Python/2.7.1 configured -- resuming normal operations
> >>> [Wed May 25 06:44:04 2011] [error] ipa: INFO: *** PROCESS START ***
> >>> [Wed May 25 06:44:04 2011] [error] ipa: INFO: *** PROCESS START ***
> >>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] mod_wsgi (pid=5192): Exception occurred processing WSGI script '/usr/share/ipa/wsgi.py'.
> >>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] Traceback (most recent call last):
> >>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] File "/usr/share/ipa/wsgi.py", line 48, in application
> >>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] return api.Backend.session(environ, start_response)
> >>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 141, in __call__
> >>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] self.create_context(ccache=environ.get('KRB5CCNAME'))
> >>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] File "/usr/lib/python2.7/site-packages/ipalib/backend.py", line 110, in create_context
> >>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] self.Backend.ldap2.connect(ccache=ccache)
> >>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] File "/usr/lib/python2.7/site-packages/ipalib/backend.py", line 62, in connect
> >>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] conn = self.create_connection(*args, **kw)
> >>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] File "/usr/lib/python2.7/site-packages/ipalib/encoder.py", line 188, in new_f
> >>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] return f(*new_args, **kwargs)
> >>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] File "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", line 337, in create_connection
> >>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] _handle_errors(e, **{})
> >>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] File "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", line 118, in _handle_errors
> >>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] raise errors.DatabaseError(desc=desc, info=info)
> >>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] DatabaseError: Local error: SASL(-1): generic failure: GSSAPI Error: An invalid name was supplied (Hostname cannot be canonicalized)
> >>> [Wed May 25 06:45:26 2011] [error] [client 10.16.78.140] mod_wsgi (pid=5193): Exception occurred processing WSGI script '/usr/share/ipa/wsgi.py'.
> >>>
> >>>
> >>> You can check the problem on vm-140.idm.lab.bos.redhat.com if you want to.
> >>>
> >>> Martin
> >>>
> >>
> >> The LDAP connection was still using the system hostname value. I added a
> >> conn.set_option(_ldap.OPT_HOST_NAME, api.env.host) in the two places we
> >> initialize an LDAP connection and that seems to have fixed it.
> >>
> >> Updated patch attached
> >>
> >> rob
> >
> > NACK. The problem on a master is gone. However, now ipa-replica-install
> > is failing:
> >
> > # ipa-replica-install /home/mkosek/replica-info-vm-027.idm.lab.bos.redhat.com.gpg
> > Directory Manager (existing master) password:
> >
> > creation of replica failed: Can't contact LDAP server:
> >
> >
> > I found out that the root cause of the failure is in the change you just
> > made in ldap2.py:
> >
> > def create_connection(self, ccache=None, bind_dn='', bind_pw='',
> > tls_cacertfile=None, tls_certfile=None, tls_keyfile=None,
> > debug_level=0):
> > ...
> > try:
> > conn = _ldap.initialize(self.ldap_uri)
> > conn.set_option(_ldap.OPT_HOST_NAME, api.env.host)<--
> > if ccache is not None:
> > os.environ['KRB5CCNAME'] = ccache
> > ...
> >
> > because api.env.host points to the local host and not the remote master.
> > When I commented this line out, installation continued OK. Then, it
> > crashed again with our "favorite" dogtag's "invalid clone_uri"
> > exception.
> >
> > Since we see this error also in other scenarios (not only custom
> > --hostname) and the root cause is not in your patch I can ACK you patch
> > 762 once the replica install bug is fixed.
> >
> > Martin
> >
>
> Fixed both of these. We only need to set the hostname when using an
> ldapi URI, so fixed both of those.
>
> I also fixed the Invalid clone_uri bug. The problem was we weren't
> passing our new hostname to pkicreate so it was creating a CA for
> whatever the value of `hostname` was. There is an environment variable
> in pkicreate to pass in the hostname and doing that has fixed the problem.
>
> rob
Yes, this issue was fixed. It's good you find a way how to deal with
clone_uri problem. However, I still hit some issues:
1) I think we have some Kerberos related problems when the custom
hostname is used (ipa.idm.lab.bos.redhat.com on a
vm-096.idm.lab.bos.redhat.com). Named and SSSD refuses to start on the
system.
/var/log/messages:
May 30 05:04:35 vm-096 named[13932]: listening on IPv4 interface eth0, 10.16.78.96#53
May 30 05:04:35 vm-096 named[13932]: generating session key for dynamic DNS
May 30 05:04:36 vm-096 named[13932]: Failed to init credentials (Preauthentication failed)
May 30 05:04:36 vm-096 named[13932]: loading configuration: failure
May 30 05:04:36 vm-096 named[13932]: exiting (due to fatal error)
May 30 05:04:36 vm-096 systemd[1]: named.service: control process exited, code=exited status=7
May 30 05:04:36 vm-096 systemd[1]: Unit named.service entered failed state.
May 30 05:07:41 vm-096 sssd: Starting up
May 30 05:07:41 vm-096 sssd[be[idm.lab.bos.redhat.com]]: Starting up
May 30 05:07:41 vm-096 sssd[be[idm.lab.bos.redhat.com]]: Error processing keytab file [(null)]: Principal [host/vm-096.idm.lab.bos.redhat.com at IDM.LAB.BOS.REDHAT.COM] was not found. Unable to create GSSAPI-encrypted LDAP connection.
2) My dogtag powered replica still refuses to install (happened to me on
2 fresh VMs) with "creation of replica failed: Configuration of CA
failed".
I investigated the ipareplica-install.log, I found a error that may be
relevant. Maybe Ade will recognize some of them.
#############################################
Attempting to connect to: vm-028.idm.lab.bos.redhat.com:9445
Connected.
Posting Query = https://vm-028.idm.lab.bos.redhat.com:9445//ca/admin/console/config/wizard?p=9&op=next&xml=true&host=vm-028.idm.lab.bos.redhat.com&port=7389&binddn=cn%3DDirectory+Manager&__bindpwd=XXXXXXXX&basedn=o%3Dipaca&database=ipaca&display=%24displayStr&cloneStartTLS=on
RESPONSE STATUS: HTTP/1.1 200 OK
RESPONSE HEADER: Server: Apache-Coyote/1.1
RESPONSE HEADER: Content-Type: application/xml;charset=UTF-8
RESPONSE HEADER: Date: Mon, 30 May 2011 11:26:29 GMT
RESPONSE HEADER: Connection: close
...
<response>
<panel>admin/console/config/databasepanel.vm</panel>
<clone>clone</clone>
<res/>
<portStr>7389</portStr>
<bindpwd>(sensitive)</bindpwd>
<cloneStartTLS>on</cloneStartTLS>
<hostname>vm-028.idm.lab.bos.redhat.com</hostname>
<errorString>Master and clone should have the same base DN</errorString>
The CA installation fails few error messages later.
Providing excerpt of CA logs as they may be relevant:
/var/log/pki-ca/catalina.out:
...
CMS Warning: FAILURE: Cannot build CA chain. Error java.security.cert.CertificateException: Certificate is not a PKCS #11 certificate|FAILURE: authz instance DirAclAuthz initialization failed and skipped, error=Property internaldb.ldapconn.port missing value|
...
[Fatal Error] :2:15: Open quote is expected for attribute "BGCOLOR" associated with an element type "BODY".
/var/log/pki-ca/system:
2893.main - [30/May/2011:07:25:47 EDT] [3] [3] Cannot build CA chain. Error java.security.cert.CertificateException: Certificate is not a PKCS #11 certificate
2893.main - [30/May/2011:07:25:47 EDT] [13] [3] authz instance DirAclAuthz initialization failed and skipped, error=Property internaldb.ldapconn.port missing value
Martin
More information about the Freeipa-devel
mailing list