[Freeipa-users] errors when one ipa server down
Rob Crittenden
rcritten at redhat.com
Mon Sep 10 13:08:07 UTC 2012
Dmitri Pal wrote:
> On 09/07/2012 04:50 PM, Rob Crittenden wrote:
>> Michael Mercier wrote:
>>>
>>> On 2012-09-07, at 2:47 PM, Dmitri Pal wrote:
>>>
>>>> On 09/07/2012 12:42 PM, Michael Mercier wrote:
>>>>> On 2012-09-07, at 12:14 PM, Dmitri Pal wrote:
>>>>>
>>>>>> On 09/06/2012 10:40 AM, Michael Mercier wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> I have experienced some odd connectivity issues using MMR with
>>>>>>> FreeIPA (all systems CentOS 6.3). I have 2 ipa servers
>>>>>>> (ipaserver / ipaserver2) setup using MMR.
>>>>>>>
>>>>>>> [root at ipaserver ~]#ipa-replica-manage list
>>>>>>> ipaserver.mpls.local: master
>>>>>>> ipaserver2.mpls.local: master
>>>>>>> [root at ipaserver ~]# rpm -qa|grep ipa
>>>>>>> libipa_hbac-1.8.0-32.el6.x86_64
>>>>>>> ipa-admintools-2.2.0-16.el6.x86_64
>>>>>>> ipa-server-2.2.0-16.el6.x86_64
>>>>>>> ipa-pki-ca-theme-9.0.3-7.el6.noarch
>>>>>>> libipa_hbac-python-1.8.0-32.el6.x86_64
>>>>>>> ipa-client-2.2.0-16.el6.x86_64
>>>>>>> ipa-server-selinux-2.2.0-16.el6.x86_64
>>>>>>> ipa-pki-common-theme-9.0.3-7.el6.noarch
>>>>>>> python-iniparse-0.3.1-2.1.el6.noarch
>>>>>>> ipa-python-2.2.0-16.el6.x86_64
>>>>>>>
>>>>>>>
>>>>>>> [root at ipaserver2 ~]#ipa-replica-manage list
>>>>>>> ipaserver.mpls.local: master
>>>>>>> ipaserver2.mpls.local: master
>>>>>>> [root at ipaserver2 ~]# rpm -qa|grep ipa
>>>>>>> ipa-client-2.2.0-16.el6.x86_64
>>>>>>> ipa-server-2.2.0-16.el6.x86_64
>>>>>>> ipa-pki-ca-theme-9.0.3-7.el6.noarch
>>>>>>> ipa-python-2.2.0-16.el6.x86_64
>>>>>>> libipa_hbac-1.8.0-32.el6.x86_64
>>>>>>> python-iniparse-0.3.1-2.1.el6.noarch
>>>>>>> libipa_hbac-python-1.8.0-32.el6.x86_64
>>>>>>> ipa-admintools-2.2.0-16.el6.x86_64
>>>>>>> ipa-server-selinux-2.2.0-16.el6.x86_64
>>>>>>> ipa-pki-common-theme-9.0.3-7.el6.noarch
>>>>>>>
>>>>>>>
>>>>>>> [mike at ipaclient ~]$ rpm -qa|grep ipa
>>>>>>> ipa-admintools-2.2.0-16.el6.x86_64
>>>>>>> python-iniparse-0.3.1-2.1.el6.noarch
>>>>>>> ipa-python-2.2.0-16.el6.x86_64
>>>>>>> libipa_hbac-python-1.8.0-32.el6.x86_64
>>>>>>> ipa-client-2.2.0-16.el6.x86_64
>>>>>>> libipa_hbac-1.8.0-32.el6.x86_64
>>>>>>>
>>>>>>>
>>>>>>> I have a webserver (zenoss) using kerberos authentication.
>>>>>>>
>>>>>>> [root at zenoss ~]# rpm -qa|grep ipa
>>>>>>> libipa_hbac-1.8.0-32.el6.x86_64
>>>>>>> libipa_hbac-python-1.8.0-32.el6.x86_64
>>>>>>> ipa-python-2.2.0-16.el6.x86_64
>>>>>>> ipa-client-2.2.0-16.el6.x86_64
>>>>>>> python-iniparse-0.3.1-2.1.el6.noarch
>>>>>>> ipa-admintools-2.2.0-16.el6.x86_64
>>>>>>>
>>>>>>> <Location />
>>>>>>> SSLRequireSSL
>>>>>>> AuthType Kerberos
>>>>>>> AuthName "Kerberos Login"
>>>>>>>
>>>>>>> KrbMethodK5Passwd Off
>>>>>>> KrbAuthRealms MPLS.LOCAL
>>>>>>> KrbSaveCredentials on
>>>>>>> KrbServiceName HTTP
>>>>>>> Krb5KeyTab /etc/http/conf.d/http.keytab
>>>>>>>
>>>>>>> AuthLDAPUrl "ldap://ipaserver.mpls.local
>>>>>>> ipaserver2.mpls.local/dc=mpls,dc=local?krbPrincipalName"
>>>>>>> RequestHeader set X_REMOTE_USER %{remoteUser}e
>>>>>>> require ldap-group
>>>>>>> cn=zenuser,cn=groups,cn=accounts,dc=mpls,dc=local
>>>>>>> </Location>
>>>>>>>
>>>>>>>
>>>>>>> With both ipaserver and ipaserver2 'up', if I connect to
>>>>>>> https://zenoss.mpls.local from ipaclient using firefox, I am
>>>>>>> successfully connected. If on ipaserver I do a 'ifdown eth0' and
>>>>>>> attempt another connection, it fails. I have also noticed the
>>>>>>> following:
>>>>>>>
>>>>>>> 1. I am unable to use the ipaserver2 management interface when
>>>>>>> ipaserver is unavailable.
>>>>>>> 2. It takes a longer period of time to do a kinit
>>>>>>>
>>>>>>> If the I then perform:
>>>>>>> [root at ipaserver ~]#ifup eth0
>>>>>>>
>>>>>>> [root at ipaserver2 ~]#ifdown eth0
>>>>>>>
>>>>>>> [mike at ipaclient ~]$kinit
>>>>>>> kinit: Cannot contact any KDC for realm 'MPLS.LOCAL' while
>>>>>>> getting initial credentials
>>>>>>>
>>>>>>> [root at ipaserver2 ~]#ifup eth0
>>>>>>>
>>>>>>> [mike at ipaclient ~]$ kinit
>>>>>>> Password for mike at MPLS.LOCAL:
>>>>>>> [mike at ipaclient ~]$
>>>>>>>
>>>>>>> [root at ipaserver2 ~]#ifdown eth0
>>>>>>>
>>>>>>> .. wait number of minutes
>>>>>>>
>>>>>>> ipaclient screen locks - type password - after a short delay (~7
>>>>>>> seconds) screen unlock compeletes
>>>>>>>
>>>>>>> [mike at ipaclient ~]$kinit
>>>>>>> Password for mike at MPLS.LOCAL:
>>>>>>> [mike at ipaclient ~]$
>>>>>>>
>>>>>>> Any ideas?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Mike
>>>>>> This seems to be some DNS problem.
>>>>>> You client does not see the second replica and might have some name
>>>>>> resolution timeouts.
>>>>>>
>>>>>> Please check your dns setup and krb5.conf on the client.
>>>>>>
>>>>>> To help more we need more details about you client configuration
>>>>>> DNS and
>>>>>> kerberos.
>>>>> Hi,
>>>>>
>>>>> Additional information...
>>>>>
>>>>> [root at zenoss ~]#more /etc/resolv.conf
>>>>> search mpls.local
>>>>> domain mpls.local
>>>>> nameserver 172.16.112.5
>>>>> nameserver 172.16.112.8
>>>>>
>>>>> [root at zenoss ~]# more /etc/krb5.conf
>>>>> #File modified by ipa-client-install
>>>>>
>>>>> [libdefaults]
>>>>> default_realm = MPLS.LOCAL
>>>>> dns_lookup_realm = true
>>>>> dns_lookup_kdc = true
>>>>> rdns = false
>>>>> ticket_lifetime = 24h
>>>>> forwardable = yes
>>>>>
>>>>> [realms]
>>>>> MPLS.LOCAL = {
>>>>> pkinit_anchors = FILE:/etc/ipa/ca.crt
>>>>> }
>>>>>
>>>>> [domain_realm]
>>>>> .mpls.local = MPLS.LOCAL
>>>>> mpls.local = MPLS.LOCAL
>>>>>
>>>>> [root at ipaclient ~]# more /etc/resolv.conf
>>>>> # Generated by NetworkManager
>>>>> search mpls.local
>>>>> nameserver 172.16.112.5
>>>>> nameserver 172.16.112.8
>>>>>
>>>>> [root at ipaclient ~]# more /etc/krb5.conf
>>>>> #File modified by ipa-client-install
>>>>>
>>>>> [libdefaults]
>>>>> default_realm = MPLS.LOCAL
>>>>> dns_lookup_realm = true
>>>>> dns_lookup_kdc = true
>>>>> rdns = false
>>>>> ticket_lifetime = 24h
>>>>> forwardable = yes
>>>>>
>>>>> [realms]
>>>>> MPLS.LOCAL = {
>>>>> pkinit_anchors = FILE:/etc/ipa/ca.crt
>>>>> }
>>>>>
>>>>> [domain_realm]
>>>>> .mpls.local = MPLS.LOCAL
>>>>> mpls.local = MPLS.LOCAL
>>>>>
>>>>> [root at ipaclient ~]# nslookup ipaserver
>>>>> Server: 172.16.112.5
>>>>> Address: 172.16.112.5#53
>>>>>
>>>>> Name: ipaserver.mpls.local
>>>>> Address: 172.16.112.5
>>>>>
>>>>> [root at ipaserver ~]#ifdown eth0
>>>>>
>>>>> [root at ipaclient ~]# nslookup ipaserver
>>>>> Server: 172.16.112.8
>>>>> Address: 172.16.112.8#53
>>>>>
>>>>> Name: ipaserver.mpls.local
>>>>> Address: 172.16.112.5
>>>>>
>>>>> [root at ipaclient ~]# nslookup ipaserver2
>>>>> Server: 172.16.112.8
>>>>> Address: 172.16.112.8#53
>>>>>
>>>>> Name: ipaserver2.mpls.local
>>>>> Address: 172.16.112.8
>>>>>
>>>>> Copy/paste from the DNS page on ipaserver/ipaserver2
>>>>>
>>>>> @ NS ipaserver.mpls.local.
>>>>> NS ipaserver2.mpls.local.
>>>>> _kerberos TXT MPLS.LOCAL
>>>>> _kerberos-master._tcp SRV 0 100 88 ipaserver
>>>>> SRV 0 100 88 ipaserver2
>>>>> _kerberos-master._udp SRV 0 100 88 ipaserver
>>>>> SRV 0 100 88 ipaserver2
>>>>> _kerberos._tcp SRV 0 100 88 ipaserver
>>>>> SRV 0 100 88 ipaserver2
>>>>> _kerberos._udp SRV 0 100 88 ipaserver
>>>>> SRV 0 100 88 ipaserver2
>>>>> _kpasswd._tcp SRV 0 100 464 ipaserver
>>>>> SRV 0 100 464 ipaserver2
>>>>> _kpasswd._udp SRV 0 100 464 ipaserver
>>>>> SRV 0 100 464 ipaserver2
>>>>> _ldap._tcp SRV 0 100 389 ipaserver
>>>>> SRV 0 100 389 ipaserver2
>>>>> _ntp._udp SRV 0 100 123 ipaserver
>>>>> SRV 0 100 123 ipaserver2
>>>>> ipaclient A 172.16.112.9
>>>>> ipaclient2 A 172.16.112.145
>>>>> ipaserver A 172.16.112.5
>>>>> ipaserver2 A 172.16.112.8
>>>>> zenoss A 172.16.112.6
>>>>>
>>>>> Thanks,
>>>>> Mike
>>>>>
>>>> I noticed that there is no domain line in the resolv.conf on the
>>>> client.
>>>> AFAIU in this case it would determine the domain by the gethostname and
>>>> in case of network being down it will fail over to the hosts file.
>>>> I wonder what is in your /etc/hosts?
>>>> Dose it have just a short host name?
>>>
>>> [root at ipaclient ~]# more /etc/hosts
>>> 127.0.0.1 localhost.localdomain localhost
>>> ::1 localhost6.localdomain6 localhost6
>>>
>>>
>>> Add domain mpls.local to /etc/resolv.conf
>>>
>>> [root at ipaserver ~]#ifdown eth0
>>>
>>> [root at ipaclient ~]# kinit mike
>>> kinit: Cannot contact any KDC for realm 'MPLS.LOCAL' while getting
>>> initial credentials
>>> [root at ipaclient ~]# nslookup ipaserver
>>> Server: 172.16.112.8
>>> Address: 172.16.112.8#53
>>>
>>> Name: ipaserver.mpls.local
>>> Address: 172.16.112.5
>>>
>>> [root at ipaclient ~]# nslookup ipaserver2
>>> Server: 172.16.112.8
>>> Address: 172.16.112.8#53
>>>
>>> Name: ipaserver2.mpls.local
>>> Address: 172.16.112.8
>>>
>>> add '172.16.112.9 ipaclient.mpls.local ipaclient' to /etc/hosts
>>>
>>> [root at ipaserver ~]#ifup eth0
>>>
>>> [root at ipaclient ~]# kinit mike
>>> Password for mike at MPLS.LOCAL:
>>>
>>> [root at ipaserver ~]#ifdown eth0
>>>
>>> [root at ipaclient ~]# kinit mike
>>> kinit: Cannot contact any KDC for realm 'MPLS.LOCAL' while getting
>>> initial credentials
>>> [root at ipaclient ~]# nslookup -type=srv _kerberos-master._tcp
>>> Server: 172.16.112.8
>>> Address: 172.16.112.8#53
>>>
>>> _kerberos-master._tcp.mpls.local service = 0 100 88
>>> ipaserver2.mpls.local.
>>> _kerberos-master._tcp.mpls.local service = 0 100 88
>>> ipaserver.mpls.local.
>>>
>>> [root at ipaclient ~]# nslookup -type=srv _kerberos-master._udp
>>> Server: 172.16.112.5
>>> Address: 172.16.112.5#53
>>>
>>> _kerberos-master._udp.mpls.local service = 0 100 88
>>> ipaserver.mpls.local.
>>> _kerberos-master._udp.mpls.local service = 0 100 88
>>> ipaserver2.mpls.local.
>>>
>>>
>>> [root at ipaclient ~]# kinit mike
>>> kinit: Cannot contact any KDC for realm 'MPLS.LOCAL' while getting
>>> initial credentials
>>>
>>> [root at ipaserver ~]#ifup eth0
>>>
>>> [root at ipaclient ~]# kinit mike
>>> Password for mike at MPLS.LOCAL:
>>
>> I'd start with the sssd logs. Is it seeing the main server go offline
>> and not switching to the second one? Or is it going into offline mode?
>>
>> Do you have _srv_ or both servers listed in ipa_server in
>> /etc/sssd/sssd.conf?
>>
>> rob
>>
> Rob, may be I am missing something but how SSSD is related in this case?
> The test is done using kinit not SSSD.
>
> It would actually be an interesting test to try the same via SSSD for
> example do su to mike instead of kinit and see what would happen (watch
> SSSD logs with high debug level, 8 for example).
> If that works it would probably mean that kinit does not fail over
> properly. So this would be a Kerberos kinit bug not IPA/SSSD bug.
>
SSSD controls the Kerberos locator. If SSSD isn't detecting that the KDC
is down then it is going to point the user to a non-working server.
rob
More information about the Freeipa-users
mailing list