[Freeipa-devel] [PATCH] 762 Let the framework be able to override the hostname
Rob Crittenden
rcritten at redhat.com
Mon Jun 6 14:51:10 UTC 2011
Martin Kosek wrote:
> On Fri, 2011-05-27 at 15:39 -0400, Rob Crittenden wrote:
>> Martin Kosek wrote:
>>> On Wed, 2011-05-25 at 11:29 -0400, Rob Crittenden wrote:
>>>> Martin Kosek wrote:
>>>>> On Fri, 2011-04-01 at 11:47 -0400, Rob Crittenden wrote:
>>>>>> The hostname is passed in during the server installation. We should use
>>>>>> this hostname for the resulting server as well. It was being discarded
>>>>>> and we always used the system hostname value.
>>>>>>
>>>>>> ticket 1052
>>>>>>
>>>>>> rob
>>>>>
>>>>> I have to NACK this again. I have a problem communicating with IPA on a
>>>>> master machine. I reproduced in on 2 different machines. Please, correct
>>>>> my steps if I am wrong, I do the following procedure
>>>>>
>>>>> 1) I prepare a fresh minimal F-15
>>>>> 2) Install freeipa-server (current master with your patches)
>>>>> 3) Add custom hostname to /etc/hosts
>>>>> 4) Install IPA server:
>>>>> ipa-server-install -p secret123 -a secret123 --hostname ipa.idm.lab.bos.redhat.com --setup-dns --forwarder=10.16.255.2
>>>>> 5) # kinit admin
>>>>> Password for admin at IDM.LAB.BOS.REDHAT.COM:
>>>>> 6) # ipa user-show admin
>>>>> ipa: ERROR: cannot connect to 'any of the configured servers':
>>>>> https://ipa.idm.lab.bos.redhat.com/ipa/xml,
>>>>> https://ipa.idm.lab.bos.redhat.com/ipa/xml
>>>>>
>>>>> # ping -c 1 ipa.idm.lab.bos.redhat.com
>>>>> PING ipa.idm.lab.bos.redhat.com (10.16.78.140) 56(84) bytes of data.
>>>>> 64 bytes from ipa.idm.lab.bos.redhat.com (10.16.78.140): icmp_req=1
>>>>> ttl=64 time=0.049 ms
>>>>>
>>>>> Apache error_log shows relevant errors:
>>>>>
>>>>> [Wed May 25 06:42:38 2011] [error] ipa: ERROR: Failed to start IPA: Unable to retrieve LDAP schema: Invalid credentials: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Permission denied)
>>>>> [Wed May 25 06:42:38 2011] [error] ipa: ERROR: Failed to start IPA: Unable to retrieve LDAP schema: Invalid credentials: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Permission denied)
>>>>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
>>>>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
>>>>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
>>>>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
>>>>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
>>>>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
>>>>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
>>>>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
>>>>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
>>>>> [Wed May 25 06:43:55 2011] [error] Exception KeyError: KeyError(140250828974112,) in<module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored
>>>>> [Wed May 25 06:43:56 2011] [notice] caught SIGTERM, shutting down
>>>>> [Wed May 25 06:43:56 2011] [notice] SELinux policy enabled; httpd running as context system_u:system_r:kernel_t:s0
>>>>> [Wed May 25 06:43:57 2011] [notice] Digest: generating secret for digest authentication ...
>>>>> [Wed May 25 06:43:57 2011] [notice] Digest: done
>>>>> [Wed May 25 06:43:57 2011] [notice] Apache/2.2.17 (Unix) DAV/2 mod_auth_kerb/5.4 mod_nss/2.2.17 NSS/3.12.9.0 mod_wsgi/3.2 Python/2.7.1 configured -- resuming normal operations
>>>>> [Wed May 25 06:44:04 2011] [error] ipa: INFO: *** PROCESS START ***
>>>>> [Wed May 25 06:44:04 2011] [error] ipa: INFO: *** PROCESS START ***
>>>>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] mod_wsgi (pid=5192): Exception occurred processing WSGI script '/usr/share/ipa/wsgi.py'.
>>>>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] Traceback (most recent call last):
>>>>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] File "/usr/share/ipa/wsgi.py", line 48, in application
>>>>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] return api.Backend.session(environ, start_response)
>>>>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 141, in __call__
>>>>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] self.create_context(ccache=environ.get('KRB5CCNAME'))
>>>>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] File "/usr/lib/python2.7/site-packages/ipalib/backend.py", line 110, in create_context
>>>>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] self.Backend.ldap2.connect(ccache=ccache)
>>>>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] File "/usr/lib/python2.7/site-packages/ipalib/backend.py", line 62, in connect
>>>>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] conn = self.create_connection(*args, **kw)
>>>>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] File "/usr/lib/python2.7/site-packages/ipalib/encoder.py", line 188, in new_f
>>>>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] return f(*new_args, **kwargs)
>>>>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] File "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", line 337, in create_connection
>>>>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] _handle_errors(e, **{})
>>>>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] File "/usr/lib/python2.7/site-packages/ipaserver/plugins/ldap2.py", line 118, in _handle_errors
>>>>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] raise errors.DatabaseError(desc=desc, info=info)
>>>>> [Wed May 25 06:45:25 2011] [error] [client 10.16.78.140] DatabaseError: Local error: SASL(-1): generic failure: GSSAPI Error: An invalid name was supplied (Hostname cannot be canonicalized)
>>>>> [Wed May 25 06:45:26 2011] [error] [client 10.16.78.140] mod_wsgi (pid=5193): Exception occurred processing WSGI script '/usr/share/ipa/wsgi.py'.
>>>>>
>>>>>
>>>>> You can check the problem on vm-140.idm.lab.bos.redhat.com if you want to.
>>>>>
>>>>> Martin
>>>>>
>>>>
>>>> The LDAP connection was still using the system hostname value. I added a
>>>> conn.set_option(_ldap.OPT_HOST_NAME, api.env.host) in the two places we
>>>> initialize an LDAP connection and that seems to have fixed it.
>>>>
>>>> Updated patch attached
>>>>
>>>> rob
>>>
>>> NACK. The problem on a master is gone. However, now ipa-replica-install
>>> is failing:
>>>
>>> # ipa-replica-install /home/mkosek/replica-info-vm-027.idm.lab.bos.redhat.com.gpg
>>> Directory Manager (existing master) password:
>>>
>>> creation of replica failed: Can't contact LDAP server:
>>>
>>>
>>> I found out that the root cause of the failure is in the change you just
>>> made in ldap2.py:
>>>
>>> def create_connection(self, ccache=None, bind_dn='', bind_pw='',
>>> tls_cacertfile=None, tls_certfile=None, tls_keyfile=None,
>>> debug_level=0):
>>> ...
>>> try:
>>> conn = _ldap.initialize(self.ldap_uri)
>>> conn.set_option(_ldap.OPT_HOST_NAME, api.env.host)<--
>>> if ccache is not None:
>>> os.environ['KRB5CCNAME'] = ccache
>>> ...
>>>
>>> because api.env.host points to the local host and not the remote master.
>>> When I commented this line out, installation continued OK. Then, it
>>> crashed again with our "favorite" dogtag's "invalid clone_uri"
>>> exception.
>>>
>>> Since we see this error also in other scenarios (not only custom
>>> --hostname) and the root cause is not in your patch I can ACK you patch
>>> 762 once the replica install bug is fixed.
>>>
>>> Martin
>>>
>>
>> Fixed both of these. We only need to set the hostname when using an
>> ldapi URI, so fixed both of those.
>>
>> I also fixed the Invalid clone_uri bug. The problem was we weren't
>> passing our new hostname to pkicreate so it was creating a CA for
>> whatever the value of `hostname` was. There is an environment variable
>> in pkicreate to pass in the hostname and doing that has fixed the problem.
>>
>> rob
>
> Yes, this issue was fixed. It's good you find a way how to deal with
> clone_uri problem. However, I still hit some issues:
>
> 1) I think we have some Kerberos related problems when the custom
> hostname is used (ipa.idm.lab.bos.redhat.com on a
> vm-096.idm.lab.bos.redhat.com). Named and SSSD refuses to start on the
> system.
>
> /var/log/messages:
> May 30 05:04:35 vm-096 named[13932]: listening on IPv4 interface eth0, 10.16.78.96#53
> May 30 05:04:35 vm-096 named[13932]: generating session key for dynamic DNS
> May 30 05:04:36 vm-096 named[13932]: Failed to init credentials (Preauthentication failed)
> May 30 05:04:36 vm-096 named[13932]: loading configuration: failure
> May 30 05:04:36 vm-096 named[13932]: exiting (due to fatal error)
> May 30 05:04:36 vm-096 systemd[1]: named.service: control process exited, code=exited status=7
> May 30 05:04:36 vm-096 systemd[1]: Unit named.service entered failed state.
> May 30 05:07:41 vm-096 sssd: Starting up
> May 30 05:07:41 vm-096 sssd[be[idm.lab.bos.redhat.com]]: Starting up
> May 30 05:07:41 vm-096 sssd[be[idm.lab.bos.redhat.com]]: Error processing keytab file [(null)]: Principal [host/vm-096.idm.lab.bos.redhat.com at IDM.LAB.BOS.REDHAT.COM] was not found. Unable to create GSSAPI-encrypted LDAP connection.
For the named issue I filed a bug against bind-dyndb-ldap for this,
https://bugzilla.redhat.com/show_bug.cgi?id=710261
This is a similar problem I ran into where when you do an ldapi bind it
defaults to using the system hostname value.
To fix the sssd problem we just need to set the ipa_hostname option
(they have lots of nice tuning options!). We just need to decide if we
always set this value or only at install time when the hostnames differ.
> 2) My dogtag powered replica still refuses to install (happened to me on
> 2 fresh VMs) with "creation of replica failed: Configuration of CA
> failed".
>
> I investigated the ipareplica-install.log, I found a error that may be
> relevant. Maybe Ade will recognize some of them.
>
> #############################################
> Attempting to connect to: vm-028.idm.lab.bos.redhat.com:9445
> Connected.
> Posting Query = https://vm-028.idm.lab.bos.redhat.com:9445//ca/admin/console/config/wizard?p=9&op=next&xml=true&host=vm-028.idm.lab.bos.redhat.com&port=7389&binddn=cn%3DDirectory+Manager&__bindpwd=XXXXXXXX&basedn=o%3Dipaca&database=ipaca&display=%24displayStr&cloneStartTLS=on
> RESPONSE STATUS: HTTP/1.1 200 OK
> RESPONSE HEADER: Server: Apache-Coyote/1.1
> RESPONSE HEADER: Content-Type: application/xml;charset=UTF-8
> RESPONSE HEADER: Date: Mon, 30 May 2011 11:26:29 GMT
> RESPONSE HEADER: Connection: close
> ...
> <response>
> <panel>admin/console/config/databasepanel.vm</panel>
> <clone>clone</clone>
> <res/>
> <portStr>7389</portStr>
> <bindpwd>(sensitive)</bindpwd>
> <cloneStartTLS>on</cloneStartTLS>
> <hostname>vm-028.idm.lab.bos.redhat.com</hostname>
> <errorString>Master and clone should have the same base DN</errorString>
>
>
> The CA installation fails few error messages later.
>
> Providing excerpt of CA logs as they may be relevant:
>
> /var/log/pki-ca/catalina.out:
> ...
> CMS Warning: FAILURE: Cannot build CA chain. Error java.security.cert.CertificateException: Certificate is not a PKCS #11 certificate|FAILURE: authz instance DirAclAuthz initialization failed and skipped, error=Property internaldb.ldapconn.port missing value|
> ...
> [Fatal Error] :2:15: Open quote is expected for attribute "BGCOLOR" associated with an element type "BODY".
>
> /var/log/pki-ca/system:
> 2893.main - [30/May/2011:07:25:47 EDT] [3] [3] Cannot build CA chain. Error java.security.cert.CertificateException: Certificate is not a PKCS #11 certificate
> 2893.main - [30/May/2011:07:25:47 EDT] [13] [3] authz instance DirAclAuthz initialization failed and skipped, error=Property internaldb.ldapconn.port missing value
>
> Martin
>
Haven't had a chance to explore this one yet. It sure would be nice if
dogtag would tell us what the two differing base DNs are though...
rob
More information about the Freeipa-devel
mailing list