[Freeipa-devel] A new proopsal for Location Based Discovery

[Simo Sorce]

Hello FreeIPA developers and other followers,

we've have thought for quite a while about how to best implement
location based discovery for our clients so that we can easily redirect
group of clients to specific servers in order to better use resources
and keep traffic local to clients.

However although we made some proposal we haven't implemented anything
so far, one reason is that all solution we came up till now were complex
and involved substantial client changes.

I recently came up with a new take on how to resolve the problem, I've
written it up here after some minimal discussion with Petr Spacek and
others.

It is available here:
http://www.freeipa.org/page/V3/DNS_Location_Mechanism


I thinks this proposal stands an actual chance at being implemented and
getting enough client support, mostly because the necessary changes to
existing clients vary from none to very minimal.

This is proposal is not yet definitive, and is open to adjustments.

I've also inlined a copy below for easier commenting.
Please trim out unnecessary text if you choose to reply and comment, and
keep only the relevant sections of text if you comment inline.

Have a good read,
Simo.

========================================================================

A Mechanism to allow for location based discovery 
by Simo Sorce with help from Petr Spacek and others


Forewords 
This is a new proposal (Jan 2013) to support Location Based discovery in
FreeIPA. It was inspired by this earlier proposal made a while ago. The
main difference is that it simplifies the whole management by
eliminating IP subnets and special client code while still maintaining a
great deal of flexibility. The key insight being that different
locations can configure the network to use different FreeIPA DNS
servers, that are local to the location being considered.


Introduction 
Service Discovery is a process whereby a client uses information stored
in the network to find servers that offer a specific service. It is
useful for two reasons. First, it places the information required for
the location of servers in the network, lessening the burden of client
configuration. Second, it gives system and network administrators a
central point of control that can be used to to define an optimized
policy that determines which clients should use which server and in what
order.

A standard for service discovery is defined in RFC 2782. This standard
defines a DNS RR with the mnemonic SRV and usage rules around it. It
allows a client to discover servers that offer a given service over a
given protocol.

A drawback of SRV Records is that it assumes clients know the correct
DNS domain to use as the query target. Without any further information,
the client's options includes using their own DNS domain and the name of
the security domain in which the client operates (such as the domain
name associated to the Kerberos REALM the client refers to). Neither
option is likely to yield optimal results however. One key service
discovery requirement, especially in large distributed enterprises, is
to choose a server that is close to a client. However in many situation
binding the client name to a specific zone that is tied to a physical
location is not possible. And even if it were it would not address the
needs of a roaming client that moves across the networks. We cannot have
the name of a client change when it move across networks as this break
the Kerberos protocol that associates keys to a fixed host name for
Service Principals.

The incompleteness of RFC 2782 is acknowledged by systems such as Active
Directory that contain extra functionality to augment SRV lookups to
make them site aware. The basic idea is to use a target in SRV service
discovery that is specific to a location or "site" in AD parlance.
Unfortunately AD clients rely on additional configureation or side
protocols to determine the client "site" and it is quite specific to
Microsoft technologies so the method they use is not easily portable and
reusable in other contexts nor documented in Standards or informative
IETF RFCs.

This document specifies a protocol for location discovery based
exclusively on DNS. While the protocol itself can be fully implemented
with a normal DNS the FreeIPA mechanism is augmented by additional
'location' awarness and will depend on additional computation done by
the FreeIPA DNS plugins.


The Discovery Protocol 
The main point about this protocol is the recognition that all we really
need is to find a specific (set of) server(s) that a specific client
needs to find. The second point is that the client has 1 bit of
information that is inequivocal: its own fully qualified name. Whether
this is fixed or assigned by the network (via DHCP) is not important,
what matters is that the name univocally identifies this specific host
in the network.

Based on these two points the idea is to make the client query for a
special set of SRV records keyed on the client's own DNS name. From the
client perspective this is the simplest protocol possible, it requires
no knowledge or hard decisions about what DNS domain name to query or
how to discover it. At the same time is allows the Domain Administrators
a lot of flexibility on how to configure these records per-client.

The failure mode for this protocol is to simply keep using the previous
heuristics, we will not define these heuristics as they are not
standardized and are implementation and deployment specific to some
extent. Suffice to say that this new protocol should not impact in any
way on previous heuristics and DNS setups and can be safely implemented
in clients with no ill effects save for an additional initial query.
Local negative chaching may help in avoiding excessive queries if the
administratoir chooses not to configure the servers to support per
client SRV Records and otherwise adds little overhead.


Client Implementation 
Because currently used SRV records are multiple and to allow the case
where a host may actually be using a domain name that is also already
used as a zone name (ie the name X.example.com identifies both an actual
host and is a subdomain where clients Y.X.example.com normally searches
for SRV records) we group all per-client location SRV records under the
_location. sub name.

So for example, a client named X.example.com would search for its own
per-client records for the LDAP service over the TCP protocol by using
the name: _ldap._tcp._location.X.example.com

With current practices a client normally looks for
_ldap._tcp.example.com instead.

It is a simple as that, the only difference between a client supporting
this new mechanism and a generic client is only about what name is used
as the 'base domain name'. Everything else is identical. Many clients
can probably be already configured to use this new base domain. And
clients that may not support it (either because the base domain is
always derived in some way and not directly configurable or because
clients refuse to use _location as a valid bade DNS name component due
to the leading '_' character) can be easily changed. Those that can't be
changed will simply fall back to use the classic SRV records on the base
domain and will simply not be location aware.

The additional advantage of using this scheme is that clients can now
use per-client SRV searches by default if they so choose because there
is no risk of ending up using unrelated servers due to unfortunate host
naming. If the administrator took the pain to configure per-client SRV
records there is an overwhelming chance those are indeed the records the
client is supposed to use. By using this as default it is possible to
make client configuration free by default which is a real boon on
networks with many hosts.

Changing defaults requires careful consideration of security
implications, please read the #Security Considerations section for more
information.


Server side implementation 
Basic solution 
The simplest way to implement this scheme on the server side is to just
create a set of records for each client. However this is a very
heavyweight and error prone process as it requires the creation of many
records for each client.


A more rational solution 
A simple but more manageable solution may be to use DNAME records as
defined by RFC 6672. The administrator in this case can set up a single
set of SRV records per location and then use a DNAME record to glue each
client to this subtree.

This solution is much more lightweight and less error prone as each
client would need one single additional record that points to a well
maintained subtree.

So a client X.example.com could have a DNAME record like this:
_location.X.example.com. DNAME Y._locations.example.com.

When the client X tries to search for its own per-client records for the
LDAP service over the TCP protocol by using the name
_ldap._tcp._location.X.example.com it would be automatically redirected
to the record _ldap._tcp.Y._locations.example.com


Advanced FreeIPA solution 
Although the above implementation works fine for most cases it has 2
major drawbacks. The first one is poor support for roaming clients as
they would be permanently referring to a specific location even when
they travel across potentially very geografically dispersed locations.
The other big drawback is that admins will have to create the DNAME
records for each client which is a lot of work. In FreeIPA we can have
more smarts given we can influence the bind-dyndb-ldap plugin behavior.

So one first very simple yet very effective simplification would be to
change the bind-dyndb-ldap plugin to create a phantom per-client
location DNAME record that points to a 'default' location.

This means DNAME records wouldn't be directly stored in LDAP but would
be synthesized by the driver if not present using a default
configuration. However to make this more useful the plugin shouldn't
just use one single default, but should have a default 'per server'.


Roaming/Remote clients 
Roaming clients or Remote clients have one big problem, although they
may have a default preferred location they move across networks and the
definition of 'location' and 'closest' server changes as they move. Yet
their name is still fixed. With a classic Bind setup this problem can
somewhat be handled by using views and changing the DNAME returned or
directly the SRV records depending on the client IP address. However
using source IP address is not always a good indicator. Clients may be
behind a NAT or maybe IP addressing is shared between multiple logical
locations within a physical network. or the client may be getting the IP
address over a VPN tunnel and so on. In general relying on IP address
information may or may not work. (There is also the minor issue that we
do not yet support views in the bind-dyndb-ldap plugin.)


Addressing the multiple locations problem 
The reason to define multiple locations is that we want to redirect
clients to different servers depending on the location they belong to.
This only really makes sense if each location has its own (set of)
FreeIPA server(s).

Also usually a location corresponds to a different network so it can be
assumed the if at least one of the FreeIPA servers in each location is a
DNS enabled server and the local network configuration (DHCP) server
serves this DNS server as the primary server for the client then we can
make the reasonable assumption that a client in a specific location will
generally query a FreeIPA server in that same location for
location-specific information.

If this holds true then changing the 'default' location base on the
server's own location would effectively make clients stick to the local
servers (Assuming the location's SRV records are properly configured to
contain only local server, which we can insure through appropriate
checks in the framework)

This is another simple optimization and works for a lot of cases but not
necessarily all. However this optimization leads to another problem.
What if the client needs to belong to a specific location indipendetly
from what server they ask to, or what if we really only have a few
FreeIPA DNS servers but want to use more locations ?

One way of course is to create a fixed DNAME record for these clients,
so the defaults do not kick in. However this is rather final. Maybe the
clients needs a preference but that preference can be overridden in some
circumstances.


Choosing the right location 
So the right location for a client may be a combination of a preference
and a set of requirements. One example of a requirement that can trump
any preference is a bandwidth constrained location.

Assume we have a client that normally resides in a large location. This
location has been segmented in small sublocations to better distribute
load so it has a preferred location. If we use a fixed DNAME to
represent this preference when this client roams to a bandwidth
constrained network it will try to use the slow link to call 'home' to
his usual location. This may be a serious problem.

However if we generate the default location dynamically we can easily
have rules on the bandwidth constrained location DNS servers that no
matter what is the preference any client asking for location based SRV
records will always be redirected to the local location which includes
only local servers in their SRV records.

This is quite powerful and would neatly solve many issues connected with
roaming clients.


DNS Slave server problem 
Dynamically choosing locations may cause issues with DNS Slaves servers,
as they wouldn't be able implement this dynamic mechanism.

One way to handle this problem is to operate in a 'degraded' mode where
DNAME records are effectively created and the location is not dynamic
per-client anymore. We can still have 'different' defaults per server if
we decide to filter DNAME records from replication. However filtering
DNAME records is also a problem because we would not be able to filter
only location based ones, it would be an all or nothing filter, which
would render DNAME records unusable for any other purpose. This
restriction is a bit extreme.

Another way might be to always compute all zone DNAME records based on
the available host records on the fly at DNS server startup, and then
keep them cached (and updated) by the bind-dyndb-ldap plugin, which will
include these records in AXFR transfers but will not write them back to
the LDAP server keeping them local. This solution might be the golden
egg, as it might allow all the advantages of dynamic generation, as well
as response performance and solve the slave server issue and perhaps
even DNSSEC related issues. It has a major drawback, it would make the
code a lot more compicated and critical.


Overall implementation proposal 
Given that the basic solution is relatively simple and require minimal
if no client changes we should consider implementing at least part of
this proposal as soon as possible. Implementing DNAME record support in
bind-dyndb-ldap seem a prerequisite and adding client support in the
SSSD IPA provider would allow to test at least with the basic setup.
This basic support should be implemented sooner rather than later so
that full dynamic support can lately be easily added to bind-dyndb-ldap
support as well as adding the necessary additional schema and UI to the
freeipa framework to mark and group clients and locations.


Security Considerations 
Client Implementation 
As always DNS replies can be spoofed relatively easily. We recommend
that SRV records resolution is used only for those clients that normally
use an additional security protocol to talk to network resources and can
use additional mechanisms to authenticate these resources. For example a
client that uses an LDAP server for security related information like
user identity information should only trust SRV record discovery for the
LDAP service if LDAPS or STARTTLS over LDAP are mandatory and
certificate verification is fully turned on, or if SASL/GSSAPI is used
with mutual authentication, integrity and confidentiality options
required. Use of DNSSEC and full DNS signature verification may be
considered an additional requirement in some cases.


Server Implementation 
TODO: interaction with DNSSEC


References 
SRV Records: RFC 2782
DNAME Records: RFC 6672


-- 
Simo Sorce * Red Hat, Inc * New York