[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Freeipa-users] errors when one ipa server down



On Mon, Sep 10, 2012 at 09:08:07AM -0400, Rob Crittenden wrote:
> Dmitri Pal wrote:
> >On 09/07/2012 04:50 PM, Rob Crittenden wrote:
> >>Michael Mercier wrote:
> >>>
> >>>On 2012-09-07, at 2:47 PM, Dmitri Pal wrote:
> >>>
> >>>>On 09/07/2012 12:42 PM, Michael Mercier wrote:
> >>>>>On 2012-09-07, at 12:14 PM, Dmitri Pal wrote:
> >>>>>
> >>>>>>On 09/06/2012 10:40 AM, Michael Mercier wrote:
> >>>>>>>Hello,
> >>>>>>>
> >>>>>>>I have experienced some odd connectivity issues using MMR with
> >>>>>>>FreeIPA (all systems CentOS 6.3).  I have 2 ipa servers
> >>>>>>>(ipaserver / ipaserver2) setup using MMR.
> >>>>>>>
> >>>>>>>[root ipaserver ~]#ipa-replica-manage list
> >>>>>>>ipaserver.mpls.local: master
> >>>>>>>ipaserver2.mpls.local: master
> >>>>>>>[root ipaserver ~]# rpm -qa|grep ipa
> >>>>>>>libipa_hbac-1.8.0-32.el6.x86_64
> >>>>>>>ipa-admintools-2.2.0-16.el6.x86_64
> >>>>>>>ipa-server-2.2.0-16.el6.x86_64
> >>>>>>>ipa-pki-ca-theme-9.0.3-7.el6.noarch
> >>>>>>>libipa_hbac-python-1.8.0-32.el6.x86_64
> >>>>>>>ipa-client-2.2.0-16.el6.x86_64
> >>>>>>>ipa-server-selinux-2.2.0-16.el6.x86_64
> >>>>>>>ipa-pki-common-theme-9.0.3-7.el6.noarch
> >>>>>>>python-iniparse-0.3.1-2.1.el6.noarch
> >>>>>>>ipa-python-2.2.0-16.el6.x86_64
> >>>>>>>
> >>>>>>>
> >>>>>>>[root ipaserver2 ~]#ipa-replica-manage list
> >>>>>>>ipaserver.mpls.local: master
> >>>>>>>ipaserver2.mpls.local: master
> >>>>>>>[root ipaserver2 ~]# rpm -qa|grep ipa
> >>>>>>>ipa-client-2.2.0-16.el6.x86_64
> >>>>>>>ipa-server-2.2.0-16.el6.x86_64
> >>>>>>>ipa-pki-ca-theme-9.0.3-7.el6.noarch
> >>>>>>>ipa-python-2.2.0-16.el6.x86_64
> >>>>>>>libipa_hbac-1.8.0-32.el6.x86_64
> >>>>>>>python-iniparse-0.3.1-2.1.el6.noarch
> >>>>>>>libipa_hbac-python-1.8.0-32.el6.x86_64
> >>>>>>>ipa-admintools-2.2.0-16.el6.x86_64
> >>>>>>>ipa-server-selinux-2.2.0-16.el6.x86_64
> >>>>>>>ipa-pki-common-theme-9.0.3-7.el6.noarch
> >>>>>>>
> >>>>>>>
> >>>>>>>[mike ipaclient ~]$ rpm -qa|grep ipa
> >>>>>>>ipa-admintools-2.2.0-16.el6.x86_64
> >>>>>>>python-iniparse-0.3.1-2.1.el6.noarch
> >>>>>>>ipa-python-2.2.0-16.el6.x86_64
> >>>>>>>libipa_hbac-python-1.8.0-32.el6.x86_64
> >>>>>>>ipa-client-2.2.0-16.el6.x86_64
> >>>>>>>libipa_hbac-1.8.0-32.el6.x86_64
> >>>>>>>
> >>>>>>>
> >>>>>>>I have a webserver (zenoss) using kerberos authentication.
> >>>>>>>
> >>>>>>>[root zenoss ~]# rpm -qa|grep ipa
> >>>>>>>libipa_hbac-1.8.0-32.el6.x86_64
> >>>>>>>libipa_hbac-python-1.8.0-32.el6.x86_64
> >>>>>>>ipa-python-2.2.0-16.el6.x86_64
> >>>>>>>ipa-client-2.2.0-16.el6.x86_64
> >>>>>>>python-iniparse-0.3.1-2.1.el6.noarch
> >>>>>>>ipa-admintools-2.2.0-16.el6.x86_64
> >>>>>>>
> >>>>>>><Location />
> >>>>>>>   SSLRequireSSL
> >>>>>>>   AuthType Kerberos
> >>>>>>>   AuthName "Kerberos Login"
> >>>>>>>
> >>>>>>>   KrbMethodK5Passwd Off
> >>>>>>>   KrbAuthRealms MPLS.LOCAL
> >>>>>>>   KrbSaveCredentials on
> >>>>>>>   KrbServiceName HTTP
> >>>>>>>   Krb5KeyTab /etc/http/conf.d/http.keytab
> >>>>>>>
> >>>>>>>   AuthLDAPUrl "ldap://ipaserver.mpls.local
> >>>>>>>ipaserver2.mpls.local/dc=mpls,dc=local?krbPrincipalName"
> >>>>>>>   RequestHeader set X_REMOTE_USER %{remoteUser}e
> >>>>>>>   require ldap-group
> >>>>>>>cn=zenuser,cn=groups,cn=accounts,dc=mpls,dc=local
> >>>>>>></Location>
> >>>>>>>
> >>>>>>>
> >>>>>>>With both ipaserver and ipaserver2 'up', if I connect to
> >>>>>>>https://zenoss.mpls.local from ipaclient using firefox, I am
> >>>>>>>successfully connected.  If on ipaserver I do a 'ifdown eth0' and
> >>>>>>>attempt another connection, it fails.  I have also noticed the
> >>>>>>>following:
> >>>>>>>
> >>>>>>>1. I am unable to use the ipaserver2 management interface when
> >>>>>>>ipaserver is unavailable.
> >>>>>>>2. It takes a longer period of time to do a kinit
> >>>>>>>
> >>>>>>>If the I then perform:
> >>>>>>>[root ipaserver ~]#ifup eth0
> >>>>>>>
> >>>>>>>[root ipaserver2 ~]#ifdown eth0
> >>>>>>>
> >>>>>>>[mike ipaclient ~]$kinit
> >>>>>>>kinit: Cannot contact any KDC for realm 'MPLS.LOCAL' while
> >>>>>>>getting initial credentials
> >>>>>>>
> >>>>>>>[root ipaserver2 ~]#ifup eth0
> >>>>>>>
> >>>>>>>[mike ipaclient ~]$ kinit
> >>>>>>>Password for mike MPLS LOCAL:
> >>>>>>>[mike ipaclient ~]$
> >>>>>>>
> >>>>>>>[root ipaserver2 ~]#ifdown eth0
> >>>>>>>
> >>>>>>>.. wait number of minutes
> >>>>>>>
> >>>>>>>ipaclient screen locks - type password - after a short delay (~7
> >>>>>>>seconds) screen unlock compeletes
> >>>>>>>
> >>>>>>>[mike ipaclient ~]$kinit
> >>>>>>>Password for mike MPLS LOCAL:
> >>>>>>>[mike ipaclient ~]$
> >>>>>>>
> >>>>>>>Any ideas?
> >>>>>>>
> >>>>>>>Thanks,
> >>>>>>>Mike
> >>>>>>This seems to be some DNS problem.
> >>>>>>You client does not see the second replica and might have some name
> >>>>>>resolution timeouts.
> >>>>>>
> >>>>>>Please check your dns setup and krb5.conf on the client.
> >>>>>>
> >>>>>>To help more we need more details about you client configuration
> >>>>>>DNS and
> >>>>>>kerberos.
> >>>>>Hi,
> >>>>>
> >>>>>Additional information...
> >>>>>
> >>>>>[root zenoss ~]#more /etc/resolv.conf
> >>>>>search mpls.local
> >>>>>domain mpls.local
> >>>>>nameserver 172.16.112.5
> >>>>>nameserver 172.16.112.8
> >>>>>
> >>>>>[root zenoss ~]# more /etc/krb5.conf
> >>>>>#File modified by ipa-client-install
> >>>>>
> >>>>>[libdefaults]
> >>>>>   default_realm = MPLS.LOCAL
> >>>>>   dns_lookup_realm = true
> >>>>>   dns_lookup_kdc = true
> >>>>>   rdns = false
> >>>>>   ticket_lifetime = 24h
> >>>>>   forwardable = yes
> >>>>>
> >>>>>[realms]
> >>>>>   MPLS.LOCAL = {
> >>>>>     pkinit_anchors = FILE:/etc/ipa/ca.crt
> >>>>>   }
> >>>>>
> >>>>>[domain_realm]
> >>>>>   .mpls.local = MPLS.LOCAL
> >>>>>   mpls.local = MPLS.LOCAL
> >>>>>
> >>>>>[root ipaclient ~]# more /etc/resolv.conf
> >>>>># Generated by NetworkManager
> >>>>>search mpls.local
> >>>>>nameserver 172.16.112.5
> >>>>>nameserver 172.16.112.8
> >>>>>
> >>>>>[root ipaclient ~]# more /etc/krb5.conf
> >>>>>#File modified by ipa-client-install
> >>>>>
> >>>>>[libdefaults]
> >>>>>   default_realm = MPLS.LOCAL
> >>>>>   dns_lookup_realm = true
> >>>>>   dns_lookup_kdc = true
> >>>>>   rdns = false
> >>>>>   ticket_lifetime = 24h
> >>>>>   forwardable = yes
> >>>>>
> >>>>>[realms]
> >>>>>   MPLS.LOCAL = {
> >>>>>     pkinit_anchors = FILE:/etc/ipa/ca.crt
> >>>>>   }
> >>>>>
> >>>>>[domain_realm]
> >>>>>   .mpls.local = MPLS.LOCAL
> >>>>>   mpls.local = MPLS.LOCAL
> >>>>>
> >>>>>[root ipaclient ~]# nslookup ipaserver
> >>>>>Server:        172.16.112.5
> >>>>>Address:    172.16.112.5#53
> >>>>>
> >>>>>Name:    ipaserver.mpls.local
> >>>>>Address: 172.16.112.5
> >>>>>
> >>>>>[root ipaserver ~]#ifdown eth0
> >>>>>
> >>>>>[root ipaclient ~]# nslookup ipaserver
> >>>>>Server:        172.16.112.8
> >>>>>Address:    172.16.112.8#53
> >>>>>
> >>>>>Name:    ipaserver.mpls.local
> >>>>>Address: 172.16.112.5
> >>>>>
> >>>>>[root ipaclient ~]# nslookup ipaserver2
> >>>>>Server:        172.16.112.8
> >>>>>Address:    172.16.112.8#53
> >>>>>
> >>>>>Name:    ipaserver2.mpls.local
> >>>>>Address: 172.16.112.8
> >>>>>
> >>>>>Copy/paste from the DNS page on ipaserver/ipaserver2
> >>>>>
> >>>>>@ NS ipaserver.mpls.local.
> >>>>>      NS ipaserver2.mpls.local.
> >>>>>_kerberos TXT MPLS.LOCAL
> >>>>>_kerberos-master._tcp SRV 0 100 88 ipaserver
> >>>>>                                          SRV 0 100 88 ipaserver2
> >>>>>_kerberos-master._udp SRV 0 100 88 ipaserver
> >>>>>                                            SRV 0 100 88 ipaserver2
> >>>>>_kerberos._tcp SRV 0 100 88 ipaserver
> >>>>>                             SRV 0 100 88 ipaserver2
> >>>>>_kerberos._udp SRV 0 100 88 ipaserver
> >>>>>                          SRV 0 100 88 ipaserver2
> >>>>>_kpasswd._tcp SRV 0 100 464 ipaserver
> >>>>>                         SRV 0 100 464 ipaserver2
> >>>>>_kpasswd._udp SRV 0 100 464 ipaserver
> >>>>>                          SRV 0 100 464 ipaserver2
> >>>>>_ldap._tcp SRV 0 100 389 ipaserver
> >>>>>                 SRV 0 100 389 ipaserver2
> >>>>>_ntp._udp SRV 0 100 123 ipaserver
> >>>>>                SRV 0 100 123 ipaserver2
> >>>>>ipaclient A 172.16.112.9
> >>>>>ipaclient2 A 172.16.112.145
> >>>>>ipaserver A 172.16.112.5
> >>>>>ipaserver2 A 172.16.112.8
> >>>>>zenoss A 172.16.112.6
> >>>>>
> >>>>>Thanks,
> >>>>>Mike
> >>>>>
> >>>>I noticed that there is no domain line in the resolv.conf on the
> >>>>client.
> >>>>AFAIU in this case it would determine the domain by the gethostname and
> >>>>in case of network being down it will fail over to the hosts file.
> >>>>I wonder what is in your /etc/hosts?
> >>>>Dose it have just a short host name?
> >>>
> >>>[root ipaclient ~]# more /etc/hosts
> >>>127.0.0.1    localhost.localdomain    localhost
> >>>::1    localhost6.localdomain6    localhost6
> >>>
> >>>
> >>>Add domain mpls.local to /etc/resolv.conf
> >>>
> >>>[root ipaserver ~]#ifdown eth0
> >>>
> >>>[root ipaclient ~]# kinit mike
> >>>kinit: Cannot contact any KDC for realm 'MPLS.LOCAL' while getting
> >>>initial credentials
> >>>[root ipaclient ~]# nslookup ipaserver
> >>>Server:        172.16.112.8
> >>>Address:    172.16.112.8#53
> >>>
> >>>Name:    ipaserver.mpls.local
> >>>Address: 172.16.112.5
> >>>
> >>>[root ipaclient ~]# nslookup ipaserver2
> >>>Server:        172.16.112.8
> >>>Address:    172.16.112.8#53
> >>>
> >>>Name:    ipaserver2.mpls.local
> >>>Address: 172.16.112.8
> >>>
> >>>add '172.16.112.9 ipaclient.mpls.local ipaclient' to /etc/hosts
> >>>
> >>>[root ipaserver ~]#ifup eth0
> >>>
> >>>[root ipaclient ~]# kinit mike
> >>>Password for mike MPLS LOCAL:
> >>>
> >>>[root ipaserver ~]#ifdown eth0
> >>>
> >>>[root ipaclient ~]# kinit mike
> >>>kinit: Cannot contact any KDC for realm 'MPLS.LOCAL' while getting
> >>>initial credentials
> >>>[root ipaclient ~]# nslookup -type=srv _kerberos-master._tcp
> >>>Server:        172.16.112.8
> >>>Address:    172.16.112.8#53
> >>>
> >>>_kerberos-master._tcp.mpls.local    service = 0 100 88
> >>>ipaserver2.mpls.local.
> >>>_kerberos-master._tcp.mpls.local    service = 0 100 88
> >>>ipaserver.mpls.local.
> >>>
> >>>[root ipaclient ~]# nslookup -type=srv _kerberos-master._udp
> >>>Server:        172.16.112.5
> >>>Address:    172.16.112.5#53
> >>>
> >>>_kerberos-master._udp.mpls.local    service = 0 100 88
> >>>ipaserver.mpls.local.
> >>>_kerberos-master._udp.mpls.local    service = 0 100 88
> >>>ipaserver2.mpls.local.
> >>>
> >>>
> >>>[root ipaclient ~]# kinit mike
> >>>kinit: Cannot contact any KDC for realm 'MPLS.LOCAL' while getting
> >>>initial credentials
> >>>
> >>>[root ipaserver ~]#ifup eth0
> >>>
> >>>[root ipaclient ~]# kinit mike
> >>>Password for mike MPLS LOCAL:
> >>
> >>I'd start with the sssd logs. Is it seeing the main server go offline
> >>and not switching to the second one? Or is it going into offline mode?
> >>
> >>Do you have _srv_ or both servers listed in ipa_server in
> >>/etc/sssd/sssd.conf?
> >>
> >>rob
> >>
> >Rob, may be I am missing something but how SSSD is related in this case?
> >The test is done using kinit not SSSD.
> >
> >It would actually be an interesting test to try the same via SSSD for
> >example do su to mike instead of kinit and see what would happen (watch
> >SSSD logs with high debug level, 8 for example).
> >If that works it would probably mean that kinit does not fail over
> >properly. So this would be a Kerberos kinit bug not IPA/SSSD bug.
> >
> 
> SSSD controls the Kerberos locator. If SSSD isn't detecting that the
> KDC is down then it is going to point the user to a non-working
> server.
> 
> rob

The SSSD only creates the file used by the locator when the first auth
request comes in trough the SSSD (in the case of IPA backed even an
identity lookup would do because it's GSSAPI-encrypted).

Bottom line, just logging in as root and performing kinit is not enough,
kinit completely bypasses the SSSD and talks to the Kerberos server
directly.


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]