[Freeipa-users] errors when one ipa server down
Dmitri Pal
dpal at redhat.com
Mon Sep 17 15:27:28 UTC 2012
On 09/17/2012 10:14 AM, Michael Mercier wrote:
> On 2012-09-07, at 4:50 PM, Rob Crittenden wrote:
>
>> Michael Mercier wrote:
>>> On 2012-09-07, at 2:47 PM, Dmitri Pal wrote:
>>>
>>>> On 09/07/2012 12:42 PM, Michael Mercier wrote:
>>>>> On 2012-09-07, at 12:14 PM, Dmitri Pal wrote:
>>>>>
>>>>>> On 09/06/2012 10:40 AM, Michael Mercier wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> I have experienced some odd connectivity issues using MMR with FreeIPA (all systems CentOS 6.3). I have 2 ipa servers (ipaserver / ipaserver2) setup using MMR.
>>>>>>>
>>>>>>> [root at ipaserver ~]#ipa-replica-manage list
>>>>>>> ipaserver.mpls.local: master
>>>>>>> ipaserver2.mpls.local: master
>>>>>>> [root at ipaserver ~]# rpm -qa|grep ipa
>>>>>>> libipa_hbac-1.8.0-32.el6.x86_64
>>>>>>> ipa-admintools-2.2.0-16.el6.x86_64
>>>>>>> ipa-server-2.2.0-16.el6.x86_64
>>>>>>> ipa-pki-ca-theme-9.0.3-7.el6.noarch
>>>>>>> libipa_hbac-python-1.8.0-32.el6.x86_64
>>>>>>> ipa-client-2.2.0-16.el6.x86_64
>>>>>>> ipa-server-selinux-2.2.0-16.el6.x86_64
>>>>>>> ipa-pki-common-theme-9.0.3-7.el6.noarch
>>>>>>> python-iniparse-0.3.1-2.1.el6.noarch
>>>>>>> ipa-python-2.2.0-16.el6.x86_64
>>>>>>>
>>>>>>>
>>>>>>> [root at ipaserver2 ~]#ipa-replica-manage list
>>>>>>> ipaserver.mpls.local: master
>>>>>>> ipaserver2.mpls.local: master
>>>>>>> [root at ipaserver2 ~]# rpm -qa|grep ipa
>>>>>>> ipa-client-2.2.0-16.el6.x86_64
>>>>>>> ipa-server-2.2.0-16.el6.x86_64
>>>>>>> ipa-pki-ca-theme-9.0.3-7.el6.noarch
>>>>>>> ipa-python-2.2.0-16.el6.x86_64
>>>>>>> libipa_hbac-1.8.0-32.el6.x86_64
>>>>>>> python-iniparse-0.3.1-2.1.el6.noarch
>>>>>>> libipa_hbac-python-1.8.0-32.el6.x86_64
>>>>>>> ipa-admintools-2.2.0-16.el6.x86_64
>>>>>>> ipa-server-selinux-2.2.0-16.el6.x86_64
>>>>>>> ipa-pki-common-theme-9.0.3-7.el6.noarch
>>>>>>>
>>>>>>>
>>>>>>> [mike at ipaclient ~]$ rpm -qa|grep ipa
>>>>>>> ipa-admintools-2.2.0-16.el6.x86_64
>>>>>>> python-iniparse-0.3.1-2.1.el6.noarch
>>>>>>> ipa-python-2.2.0-16.el6.x86_64
>>>>>>> libipa_hbac-python-1.8.0-32.el6.x86_64
>>>>>>> ipa-client-2.2.0-16.el6.x86_64
>>>>>>> libipa_hbac-1.8.0-32.el6.x86_64
>>>>>>>
>>>>>>>
>>>>>>> I have a webserver (zenoss) using kerberos authentication.
>>>>>>>
>>>>>>> [root at zenoss ~]# rpm -qa|grep ipa
>>>>>>> libipa_hbac-1.8.0-32.el6.x86_64
>>>>>>> libipa_hbac-python-1.8.0-32.el6.x86_64
>>>>>>> ipa-python-2.2.0-16.el6.x86_64
>>>>>>> ipa-client-2.2.0-16.el6.x86_64
>>>>>>> python-iniparse-0.3.1-2.1.el6.noarch
>>>>>>> ipa-admintools-2.2.0-16.el6.x86_64
>>>>>>>
>>>>>>> <Location />
>>>>>>> SSLRequireSSL
>>>>>>> AuthType Kerberos
>>>>>>> AuthName "Kerberos Login"
>>>>>>>
>>>>>>> KrbMethodK5Passwd Off
>>>>>>> KrbAuthRealms MPLS.LOCAL
>>>>>>> KrbSaveCredentials on
>>>>>>> KrbServiceName HTTP
>>>>>>> Krb5KeyTab /etc/http/conf.d/http.keytab
>>>>>>>
>>>>>>> AuthLDAPUrl "ldap://ipaserver.mpls.local ipaserver2.mpls.local/dc=mpls,dc=local?krbPrincipalName"
>>>>>>> RequestHeader set X_REMOTE_USER %{remoteUser}e
>>>>>>> require ldap-group cn=zenuser,cn=groups,cn=accounts,dc=mpls,dc=local
>>>>>>> </Location>
>>>>>>>
>>>>>>>
>>>>>>> With both ipaserver and ipaserver2 'up', if I connect to https://zenoss.mpls.local from ipaclient using firefox, I am successfully connected. If on ipaserver I do a 'ifdown eth0' and attempt another connection, it fails. I have also noticed the following:
>>>>>>>
>>>>>>> 1. I am unable to use the ipaserver2 management interface when ipaserver is unavailable.
>>>>>>> 2. It takes a longer period of time to do a kinit
>>>>>>>
>>>>>>> If the I then perform:
>>>>>>> [root at ipaserver ~]#ifup eth0
>>>>>>>
>>>>>>> [root at ipaserver2 ~]#ifdown eth0
>>>>>>>
>>>>>>> [mike at ipaclient ~]$kinit
>>>>>>> kinit: Cannot contact any KDC for realm 'MPLS.LOCAL' while getting initial credentials
>>>>>>>
>>>>>>> [root at ipaserver2 ~]#ifup eth0
>>>>>>>
>>>>>>> [mike at ipaclient ~]$ kinit
>>>>>>> Password for mike at MPLS.LOCAL:
>>>>>>> [mike at ipaclient ~]$
>>>>>>>
>>>>>>> [root at ipaserver2 ~]#ifdown eth0
>>>>>>>
>>>>>>> .. wait number of minutes
>>>>>>>
>>>>>>> ipaclient screen locks - type password - after a short delay (~7 seconds) screen unlock compeletes
>>>>>>>
>>>>>>> [mike at ipaclient ~]$kinit
>>>>>>> Password for mike at MPLS.LOCAL:
>>>>>>> [mike at ipaclient ~]$
>>>>>>>
>>>>>>> Any ideas?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Mike
>>>>>> This seems to be some DNS problem.
>>>>>> You client does not see the second replica and might have some name
>>>>>> resolution timeouts.
>>>>>>
>>>>>> Please check your dns setup and krb5.conf on the client.
>>>>>>
>>>>>> To help more we need more details about you client configuration DNS and
>>>>>> kerberos.
>>>>> Hi,
>>>>>
>>>>> Additional information...
>>>>>
>>>>> [root at zenoss ~]#more /etc/resolv.conf
>>>>> search mpls.local
>>>>> domain mpls.local
>>>>> nameserver 172.16.112.5
>>>>> nameserver 172.16.112.8
>>>>>
>>>>> [root at zenoss ~]# more /etc/krb5.conf
>>>>> #File modified by ipa-client-install
>>>>>
>>>>> [libdefaults]
>>>>> default_realm = MPLS.LOCAL
>>>>> dns_lookup_realm = true
>>>>> dns_lookup_kdc = true
>>>>> rdns = false
>>>>> ticket_lifetime = 24h
>>>>> forwardable = yes
>>>>>
>>>>> [realms]
>>>>> MPLS.LOCAL = {
>>>>> pkinit_anchors = FILE:/etc/ipa/ca.crt
>>>>> }
>>>>>
>>>>> [domain_realm]
>>>>> .mpls.local = MPLS.LOCAL
>>>>> mpls.local = MPLS.LOCAL
>>>>>
>>>>> [root at ipaclient ~]# more /etc/resolv.conf
>>>>> # Generated by NetworkManager
>>>>> search mpls.local
>>>>> nameserver 172.16.112.5
>>>>> nameserver 172.16.112.8
>>>>>
>>>>> [root at ipaclient ~]# more /etc/krb5.conf
>>>>> #File modified by ipa-client-install
>>>>>
>>>>> [libdefaults]
>>>>> default_realm = MPLS.LOCAL
>>>>> dns_lookup_realm = true
>>>>> dns_lookup_kdc = true
>>>>> rdns = false
>>>>> ticket_lifetime = 24h
>>>>> forwardable = yes
>>>>>
>>>>> [realms]
>>>>> MPLS.LOCAL = {
>>>>> pkinit_anchors = FILE:/etc/ipa/ca.crt
>>>>> }
>>>>>
>>>>> [domain_realm]
>>>>> .mpls.local = MPLS.LOCAL
>>>>> mpls.local = MPLS.LOCAL
>>>>>
>>>>> [root at ipaclient ~]# nslookup ipaserver
>>>>> Server: 172.16.112.5
>>>>> Address: 172.16.112.5#53
>>>>>
>>>>> Name: ipaserver.mpls.local
>>>>> Address: 172.16.112.5
>>>>>
>>>>> [root at ipaserver ~]#ifdown eth0
>>>>>
>>>>> [root at ipaclient ~]# nslookup ipaserver
>>>>> Server: 172.16.112.8
>>>>> Address: 172.16.112.8#53
>>>>>
>>>>> Name: ipaserver.mpls.local
>>>>> Address: 172.16.112.5
>>>>>
>>>>> [root at ipaclient ~]# nslookup ipaserver2
>>>>> Server: 172.16.112.8
>>>>> Address: 172.16.112.8#53
>>>>>
>>>>> Name: ipaserver2.mpls.local
>>>>> Address: 172.16.112.8
>>>>>
>>>>> Copy/paste from the DNS page on ipaserver/ipaserver2
>>>>>
>>>>> @ NS ipaserver.mpls.local.
>>>>> NS ipaserver2.mpls.local.
>>>>> _kerberos TXT MPLS.LOCAL
>>>>> _kerberos-master._tcp SRV 0 100 88 ipaserver
>>>>> SRV 0 100 88 ipaserver2
>>>>> _kerberos-master._udp SRV 0 100 88 ipaserver
>>>>> SRV 0 100 88 ipaserver2
>>>>> _kerberos._tcp SRV 0 100 88 ipaserver
>>>>> SRV 0 100 88 ipaserver2
>>>>> _kerberos._udp SRV 0 100 88 ipaserver
>>>>> SRV 0 100 88 ipaserver2
>>>>> _kpasswd._tcp SRV 0 100 464 ipaserver
>>>>> SRV 0 100 464 ipaserver2
>>>>> _kpasswd._udp SRV 0 100 464 ipaserver
>>>>> SRV 0 100 464 ipaserver2
>>>>> _ldap._tcp SRV 0 100 389 ipaserver
>>>>> SRV 0 100 389 ipaserver2
>>>>> _ntp._udp SRV 0 100 123 ipaserver
>>>>> SRV 0 100 123 ipaserver2
>>>>> ipaclient A 172.16.112.9
>>>>> ipaclient2 A 172.16.112.145
>>>>> ipaserver A 172.16.112.5
>>>>> ipaserver2 A 172.16.112.8
>>>>> zenoss A 172.16.112.6
>>>>>
>>>>> Thanks,
>>>>> Mike
>>>>>
>>>> I noticed that there is no domain line in the resolv.conf on the client.
>>>> AFAIU in this case it would determine the domain by the gethostname and
>>>> in case of network being down it will fail over to the hosts file.
>>>> I wonder what is in your /etc/hosts?
>>>> Dose it have just a short host name?
>>> [root at ipaclient ~]# more /etc/hosts
>>> 127.0.0.1 localhost.localdomain localhost
>>> ::1 localhost6.localdomain6 localhost6
>>>
>>>
>>> Add domain mpls.local to /etc/resolv.conf
>>>
>>> [root at ipaserver ~]#ifdown eth0
>>>
>>> [root at ipaclient ~]# kinit mike
>>> kinit: Cannot contact any KDC for realm 'MPLS.LOCAL' while getting initial credentials
>>> [root at ipaclient ~]# nslookup ipaserver
>>> Server: 172.16.112.8
>>> Address: 172.16.112.8#53
>>>
>>> Name: ipaserver.mpls.local
>>> Address: 172.16.112.5
>>>
>>> [root at ipaclient ~]# nslookup ipaserver2
>>> Server: 172.16.112.8
>>> Address: 172.16.112.8#53
>>>
>>> Name: ipaserver2.mpls.local
>>> Address: 172.16.112.8
>>>
>>> add '172.16.112.9 ipaclient.mpls.local ipaclient' to /etc/hosts
>>>
>>> [root at ipaserver ~]#ifup eth0
>>>
>>> [root at ipaclient ~]# kinit mike
>>> Password for mike at MPLS.LOCAL:
>>>
>>> [root at ipaserver ~]#ifdown eth0
>>>
>>> [root at ipaclient ~]# kinit mike
>>> kinit: Cannot contact any KDC for realm 'MPLS.LOCAL' while getting initial credentials
>>> [root at ipaclient ~]# nslookup -type=srv _kerberos-master._tcp
>>> Server: 172.16.112.8
>>> Address: 172.16.112.8#53
>>>
>>> _kerberos-master._tcp.mpls.local service = 0 100 88 ipaserver2.mpls.local.
>>> _kerberos-master._tcp.mpls.local service = 0 100 88 ipaserver.mpls.local.
>>>
>>> [root at ipaclient ~]# nslookup -type=srv _kerberos-master._udp
>>> Server: 172.16.112.5
>>> Address: 172.16.112.5#53
>>>
>>> _kerberos-master._udp.mpls.local service = 0 100 88 ipaserver.mpls.local.
>>> _kerberos-master._udp.mpls.local service = 0 100 88 ipaserver2.mpls.local.
>>>
>>>
>>> [root at ipaclient ~]# kinit mike
>>> kinit: Cannot contact any KDC for realm 'MPLS.LOCAL' while getting initial credentials
>>>
>>> [root at ipaserver ~]#ifup eth0
>>>
>>> [root at ipaclient ~]# kinit mike
>>> Password for mike at MPLS.LOCAL:
>> I'd start with the sssd logs. Is it seeing the main server go offline and not switching to the second one? Or is it going into offline mode?
>>
>> Do you have _srv_ or both servers listed in ipa_server in /etc/sssd/sssd.conf?
>>
> Hello,
>
> [root at ipaclient ~]# more /etc/sssd/sssd.conf
> [sssd]
> config_file_version = 2
> services = nss, pam
> # SSSD will not start if you do not configure any domains.
> # Add new domain configurations as [domain/<NAME>] sections, and
> # then add the list of domains (in the order you want them to be
> # queried) to the "domains" attribute below and uncomment it.
> # domains = LDAP
>
> domains = mpls.local
> [nss]
>
> [pam]
>
> # Example LDAP domain
> # [domain/LDAP]
> # id_provider = ldap
> # auth_provider = ldap
> # ldap_schema can be set to "rfc2307", which stores group member names in the
> # "memberuid" attribute, or to "rfc2307bis", which stores group member DNs in
> # the "member" attribute. If you do not know this value, ask your LDAP
> # administrator.
> # ldap_schema = rfc2307
> # ldap_uri = ldap://ldap.mydomain.org
> # ldap_search_base = dc=mydomain,dc=org
> # Note that enabling enumeration will have a moderate performance impact.
> # Consequently, the default value for enumeration is FALSE.
> # Refer to the sssd.conf man page for full details.
> # enumerate = false
> # Allow offline logins by locally storing password hashes (default: false).
> # cache_credentials = true
>
> # An example Active Directory domain. Please note that this configuration
> # works for AD 2003R2 and AD 2008, because they use pretty much RFC2307bis
> # compliant attribute names. To support UNIX clients with AD 2003 or older,
> # you must install Microsoft Services For Unix and map LDAP attributes onto
> # msSFU30* attribute names.
> # [domain/AD]
> # id_provider = ldap
> # auth_provider = krb5
> # chpass_provider = krb5
> #
> # ldap_uri = ldap://your.ad.example.com
> # ldap_search_base = dc=example,dc=com
> # ldap_schema = rfc2307bis
> # ldap_sasl_mech = GSSAPI
> # ldap_user_object_class = user
> # ldap_group_object_class = group
> # ldap_user_home_directory = unixHomeDirectory
> # ldap_user_principal = userPrincipalName
> # ldap_account_expire_policy = ad
> # ldap_force_upper_case_realm = true
> #
> # krb5_server = your.ad.example.com
> # krb5_realm = EXAMPLE.COM
> [domain/mpls.local]
> cache_credentials = True
> krb5_store_password_if_offline = True
> ipa_domain = mpls.local
> id_provider = ipa
> auth_provider = ipa
> access_provider = ipa
> chpass_provider = ipa
> ipa_dyndns_update = True
> ipa_server = _srv_, ipaserver.mpls.local, ipaserver2.mpls.local
Can you please for the sake of the test remove _srv_ from your
configuration?
There might be a bug in how we handle the case when the response from
DNS lookup is not obtained or something like.
It seems that it does not fail over properly.
> ldap_tls_cacert = /etc/ipa/ca.crt
>
> NOTE: I manually added ipaserver2.mpls.local
>
> Where specifically should I add the debugging?
> I added debug_level = 5 to [sssd]
You can add it to the bottom. That should work.
> [root at ipaserver ~]ifdown eth0
>
> [root at ipaserver2 ~]ifup eth0
>
> (Mon Sep 17 10:08:47 2012) [sssd] [ping_check] (0x0100): Service mpls.local replied to ping
> (Mon Sep 17 10:08:48 2012) [sssd] [service_send_ping] (0x0100): Pinging nss
> (Mon Sep 17 10:08:48 2012) [sssd] [service_send_ping] (0x0100): Pinging pam
> (Mon Sep 17 10:08:48 2012) [sssd] [ping_check] (0x0100): Service nss replied to ping
> (Mon Sep 17 10:08:48 2012) [sssd] [ping_check] (0x0100): Service pam replied to ping
> (Mon Sep 17 10:08:57 2012) [sssd] [service_send_ping] (0x0100): Pinging mpls.local
> (Mon Sep 17 10:08:57 2012) [sssd] [ping_check] (0x0100): Service mpls.local replied to ping
> (Mon Sep 17 10:08:58 2012) [sssd] [service_send_ping] (0x0100): Pinging nss
> (Mon Sep 17 10:08:58 2012) [sssd] [service_send_ping] (0x0100): Pinging pam
> (Mon Sep 17 10:08:58 2012) [sssd] [ping_check] (0x0100): Service nss replied to ping
> (Mon Sep 17 10:08:58 2012) [sssd] [ping_check] (0x0100): Service pam replied to ping
> (Mon Sep 17 10:09:07 2012) [sssd] [service_send_ping] (0x0100): Pinging mpls.local
> (Mon Sep 17 10:09:07 2012) [sssd] [ping_check] (0x0100): Service mpls.local replied to ping
> (Mon Sep 17 10:09:08 2012) [sssd] [service_send_ping] (0x0100): Pinging nss
> (Mon Sep 17 10:09:08 2012) [sssd] [service_send_ping] (0x0100): Pinging pam
> (Mon Sep 17 10:09:08 2012) [sssd] [ping_check] (0x0100): Service nss replied to ping
> (Mon Sep 17 10:09:08 2012) [sssd] [ping_check] (0x0100): Service pam replied to ping
> (Mon Sep 17 10:09:17 2012) [sssd] [service_send_ping] (0x0100): Pinging mpls.local
> (Mon Sep 17 10:09:17 2012) [sssd] [ping_check] (0x0100): Service mpls.local replied to ping
> (Mon Sep 17 10:09:18 2012) [sssd] [service_send_ping] (0x0100): Pinging nss
> (Mon Sep 17 10:09:18 2012) [sssd] [service_send_ping] (0x0100): Pinging pam
> (Mon Sep 17 10:09:18 2012) [sssd] [ping_check] (0x0100): Service nss replied to ping
> (Mon Sep 17 10:09:18 2012) [sssd] [ping_check] (0x0100): Service pam replied to ping
>
This is not the right log. The most informative one is called
sssd_default.log.
>
>> rob
>>
--
Thank you,
Dmitri Pal
Sr. Engineering Manager for IdM portfolio
Red Hat Inc.
-------------------------------
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/
More information about the Freeipa-users
mailing list