Chapter 5

Distributing Your Data

Distributing your data allows you to scale your directory across multiple servers without physically containing those directory entries on each server in your enterprise. A distributed directory can thus hold a much larger number of entries than would be possible with a single server.

In addition, you can configure your directory to hide the distributing details from the user. As far as users and applications are concerned, there is simply a single directory that answers their directory queries.

The following sections describe the mechanics of data distribution in more detail:

About Using Multiple Databases

Directory Server stores data in LDBM databases.The LDBM database is a high-performance disk-based database. Each database consists of a set of large files that contains all of the data assigned to it.

You can store different portions of your directory tree in different databases. In that instance, your directory tree appears as follows:

You can also store the data of the three suffixes in three separate databases, as shown below:

When you divide your directory tree among a number of databases, these databases can then be distributed across multiple servers. For example, the three databases you created to contain the three suffixes of your directory tree can be stored on two servers, as follows:

Server A contains databases one and two, and server B contains database three.

Distributing databases across multiple servers reduces the amount of work each server needs to do. Thus, the directory can be made to scale to a much larger number of entries than would be possible with a single server.

In addition, Directory Server supports adding databases dynamically, meaning you can add new databases when your directory needs them without taking your entire directory off-line.

About Suffixes

Each database contains the data within a suffix of your Directory Server. You can create both root and sub suffixes to organize the contents of your directory tree. A root suffix is the entry at the top of a tree. It can be the root of your directory tree or part of a larger tree you have designed for your Directory Server.

A sub suffix is a branch underneath a root suffix. The data for root and sub suffixes are contained by databases.

Assume, for example, you want to create suffixes to represent the distribution of your directory data. The directory tree for example.com Corporation appears as the following:

If example.com Corporation decided to split their directory tree across five different databases, the new tree would appear as this:

The suffixes that result contain the following entries:

The o=NetscapeRoot and dc=example,dc=com suffixes are both root suffixes. The other ou=testing,dc=example,dc=com suffix, the ou=development,dc=example,dc=com suffix and the ou=partners,ou=development,dc=example,dc=com suffix are all sub suffixes of the dc=example,dc=com root suffix. The root suffix dc=example,dc=com contains the data in the ou=marketing branch of the original directory tree.

Your directory might contain more than one root suffix. For example, an ISP called Example might host several websites, one for example_a.com and one for example_b.com. The ISP would create two root suffixes, one corresponding to the o=example_a.com naming context and one corresponding to the o=example_b.com naming context. The directory tree appears as follows:

The dc=example,dc=com entry represents a root suffix. The entry for each hosted ISP is also a root suffix (o=example_a.com and o=example_b.com). The ou=people and the ou=groups branches are sub suffixes under each root suffix.

About Knowledge References

Once you have distributed your data over several databases, you need to define the relationship between the distributed data. You do this using knowledge references, pointers to directory information held in different databases. The Directory Server provides the following types of knowledge references to help you link your distributed data into a single directory tree:

Referrals -- The server returns a piece of information to the client application indicating that the client application needs to contact another server to fulfill the request.
Chaining -- The server contacts other servers on behalf of the client application and returns the combined results to the client application after finishing the operation.

The following sections describe and compare these two types of knowledge references in more detail.

Using Referrals

A referral is a piece of information returned by a server that tells a client application which server to contact to proceed with an operation request. This redirection mechanism occurs when a client application requests a directory entry that does not exist on the local server.

Directory Server supports two types of referrals:

A default referral -- The directory returns a default referral when a client application presents a DN for which the server does not have a matching suffix. Default referrals are stored in the configuration file of the server. You can set a default referral for the Directory Server and a separate default referral for each database.

The default referral you set for each database is done through the suffix configuration information. When the suffix of the database is disabled, you can configure the directory to return a default referral to client requests made to that suffix. For more information about suffixes, refer to About Suffixes. For information on configuring suffixes, refer to Netscape Directory Server Administrator's Guide.

Smart referrals -- Smart referrals are stored on entries within the directory itself. Smart referrals point to Directory Servers that have knowledge of the subtree whose DN matches the DN of the entry containing the smart referral.

All referrals are returned in the format of an LDAP uniform resource locator (URL). The following sections describe the structure of an LDAP referral, and then describe the two referral types supported by Directory Server.

The Structure of an LDAP Referral

An LDAP referral contains information in the format of an LDAP URL. An LDAP URL contains the following information:

The host name of the server to contact
The port number of the server
The base DN (for search operations) or target DN (for add, delete, and modify operations).

For example, a client application searches dc=example,dc=com for entries with a surname Jensen. A referral returns the following LDAP URL to the client application:

ldap://europe.example.com:389/ou=people,l=europe,dc=example,dc=com

The referral tells the client application to contact the host europe.example.com on port 389 and submit a search rooted at ou=people,l=europe,dc=example,dc=com.

The LDAP client application you use determines how a referral is handled. Some client applications automatically retry the operation on the server to which they have been referred. Other client applications simply return the referral information to the user. Most LDAP client applications provided by Netscape (such as the command-line utilities or the Directory Server Gateway) automatically follow the referral. The same bind credentials you supply on the initial directory request are used to access the server.

Most client applications follow a limited number of referrals, or hops. The limit on the number of referrals followed reduces the time a client application spends trying to complete a directory lookup request and helps eliminate hung processes caused by circular referral patterns.

About Default Referrals

Default referrals are returned to clients when the server or database contacted does not contain the data requested.

Directory Server determines whether a default referral should be returned by comparing the DN of the requested directory object against the directory suffixes supported by the local server. If the DN does not match the supported suffixes, the Directory Server returns a default referral.

For example, a directory client requests the following directory entry:

uid=bjensen,ou=people,dc=example,dc=com

However, the server manages only entries stored under the dc=europe,dc=example,dc=com suffix. The directory returns a referral to the client that indicates which server to contact for entries stored in the dc=example,dc=com suffix. The client then contacts the appropriate server and resubmits the original request.

You configure the default referral to point to a Directory Server that has more knowledge about the distribution of your directory. Default referrals for the server are set by the nsslapd-referral attribute. Default referrals for each database in your directory installation are set by the nsslapd-referral attribute in the database entry in the configuration. These attribute values are stored in the dse.ldif file.

For information on configuring default referrals, see the Netscape Directory Server Administrator's Guide.

Smart Referrals

Directory Server also allows you to configure your directory to use smart referrals. Smart referrals allow you to associate a directory entry or directory tree to a specific LDAP URL. Associating directory entries to specific LDAP URLs allows you to refer requests to any of the following:

Same namespace contained on a different server
Different namespaces on a local server
Different namespaces on the same server

Unlike default referrals, the smart referrals are stored within the directory itself. For information on configuring and managing smart referrals, refer to the Netscape Directory Server Administrator's Guide.

For example, the directory for the American office of example.com Corporation contains the following directory branch point: ou=people,dc=example,dc=com.

You redirect all requests on this branch to the ou=people branch of the European office of example.com Corporation by specifying a smart referral on the ou=people entry itself. This smart referral appears as follows:

ldap://europe.example.com:389/ou=people,dc=example,dc=com

Any requests made to people branch of the American directory are redirected to the European directory. An illustration of this smart referral follows:

You can use the same mechanism to redirect queries to a different server that uses a different namespace. For example, an employee working in the Italian office of example.com Corporation makes a request to the European directory for the phone number of an example.com employee in America. The referral returned by the directory follows:

ldap://europe.example.com:389/ou=US employees,dc=example,dc=com

The following diagram illustrates how the referral works:

Finally, if you serve multiple suffixes on the same server, you can redirect queries from one namespace to another namespace served on the same machine. If you want to redirect all queries on the local machine for o=example,c=us to dc=example,dc=com, then you would put the following smart referral on the o=example,c=us entry:

ldap:///dc=example,dc=com

The third slash in this LDAP URL indicates that the URL points to the same Directory Server.


Note	Creating a referral from one namespace to another works only for clients whose searches are based at that distinguished name. Other kinds of operations, such as searches below ou=people,o=example,c=US, will not be performed correctly.

For more information on LDAP URLS and on how to include smart URLs on Directory Server entries, see Netscape Directory Server Administrator's Guide.

Tips for Designing Smart Referrals

While simple to implement, consider the following points before using smart referrals:

Keep the design simple.

Deploying your directory using a complex web of referrals makes administration difficult. Also, overusing smart referrals can lead to circular referral patterns. For example, a referral points to an LDAP URL, this LDAP URL in turn points to another LDAP URL, and so on until a referral somewhere in the chain points back to the original server. The following diagram depicts a circular referral pattern:

Redirect at major branch points.

Limit your referral usage to handle redirection at the suffix level of your directory tree. Smart referrals allow you to redirect lookup requests for leaf (non-branch) entries to different servers and DNs. As a result, you may be tempted to use smart referrals as an aliasing mechanism, leading to a complex and difficult method to secure directory structure. By limiting referrals to the suffix or major branch points of your directory tree, you can limit the number of referrals that you have to manage, subsequently reducing your directory's administrative overhead.

Consider the security implications.

Access control does not cross referral boundaries. Even if the server where the request originated allows access to an entry, when a smart referral sends a client request to another server, the client application may not be allowed access.

Also, the client credentials need to be available on the server to which the client is referred for client authentication to take place.

Using Chaining

Chaining is a method for relaying requests to another server. This method is implemented through database links. A database link, as described in the section titled Distributing Your Data, contains no data. Instead, it redirects client application requests to remote servers that contain the data.

During chaining, a server receives a request from a client application for data it does not contain. Using the database link, the server then contacts other servers on behalf of the client application and returns the results to the client application. This operation is illustrated in the following diagram.

Each database link is associated to a remote server holding data. You can also configure alternate remote servers containing replicas of the data for the database link to use when there is a failure. For more information on configuring database links, refer to the Netscape Directory Server Administrator's Guide.

Database links provide the following features:

Invisible access to remote data.

Because the database link takes care of client requests, data distribution is completely hidden from the client.

Dynamic management.

You can add or remove a part of the directory from the system while the entire system remains available to client applications. The database link can temporarily return referrals to the application until entries have been redistributed across the directory. You can also implement this functionality through the suffix itself, which can return a referral rather than forwarding a client application to the database.

Access control.

The database link impersonates the client application, providing the appropriate authorization identity to the remote server. You can disable user impersonation on the remote servers when access control evaluation is not required. For more information on configuring database links, refer to the Netscape Directory Server Administrator's Guide.

Note

If chaining is configured between a current version multiplexor and a 4.x farm server, add the nsuniqueid attribute to the 4.x farm server schema. If this attribute is not added to the 4.x Directory Server schema, the current version multiplexor will not find the entry it expects, and chaining will fail.

To add the attribute type to the 4.x schema, add the following line to the 4.x farm server's slapd-user_at.conf file under the serverRoot/slapd-serverID/config directory:

attribute nsuniqueid nsuniqueid 2.16.840.1.113730.3.1.542 int single operational

Deciding Between Referrals and Chaining

Both methods of linking your directory partitions have advantages and disadvantages. The method, or combination of methods, you choose depends upon the specific needs of your directory.

The major difference between the two knowledge references is the location of the intelligence that knows how to locate the distributed information. In a chained system, the intelligence is implemented in the servers. In a system that uses referrals, the intelligence is implemented in the client application.

While chaining reduces client complexity, it does so at the cost of increased server complexity. Chained servers must work with remote servers and send the results to directory clients.

With referrals, the client must handle locating the referral and collating search results. However, referrals offer more flexibility for the writers of client applications and allow developers to provide better feedback to users about the progress of a distributed directory operation.

The following sections describe some of the more specific differences between referrals and chaining in greater detail.

Usage Differences

Some client applications do not support referrals. Chaining allows client applications to communicate with a single server and still access the data stored on many servers. Sometimes referrals do not work when a company's network uses proxies. For example, a client application has permissions to speak to only one server inside a firewall. If they are referred to a different server, they will not be able to contact it successfully.

Also, with referrals a client must authenticate, meaning that the servers to which clients are being referred need to contain the client credentials. With chaining, client authentication takes place only once. Clients do not need to authenticate again on the servers to which their requests are chained.

Evaluating Access Controls

Chaining evaluates access controls differently from referrals. With referrals, an entry for the client must exist on all of the target servers. With chaining, the client entry does not need to be on all of the target servers.

For example, a client sends a search request to server A. The following diagram illustrates the operation using referrals:

In the illustration above, the client application performs the following steps:

The client application first binds with server A.
Server A contains an entry for the client that provides a user name and password, so it returns a bind acceptance message. In order for the referral to work, the client entry must be present on server A.
The client application sends the operation request to server A.
However, server A does not contain the information requested. Instead, server A returns a referral to the client application telling them to contact server B.
The client application then sends a bind request to server B. To bind successfully, server B must also contain an entry for the client application.
The bind is successful, and the client application can now resubmit its search operation to server B.

This approach requires server B to have a replicated copy of the client's entry from server A.

Chaining solves this problem. A search request would occur as follows on a chained system:

In the illustration above, the following steps are performed:

The client application binds with server A, and server A tries to confirm that the user name and password are correct.
Server A does not contain an entry corresponding to the client application. Instead, it contains a database link to server B, which contains the actual entry of the client. Server A sends a bind request to server B.
Server B sends an acceptance response to server A.
Server A then processes the client application's request using the database link. The database link contacts a remote data store located on server B to process the search operation.

In a chained system, the entry corresponding to the client application does not need to be located on the same server as the data the client requests. For example, a system could be set up as follows:

In this illustration, the following steps are performed:

The client application binds with server A, and server A tries to confirm that the user name and password are correct.
Server A does not contain an entry corresponding to the client application. Instead, it contains a database link to server B, which contains the actual entry of the client. Server A sends a bind request to server B.
Server B sends an acceptance response to server A.
Server A then processes the client application's request using another database link. The database link contacts a remote data store located on server C to process the search operation.

However, database links do not support the following access controls:

Controls that must access the content of the user entry are not supported when the user entry is located on a different server. This includes access controls based on groups, filters, and roles.
Controls based on client IP addresses or DNS domains may be denied. This is because the database link impersonates the client when it contacts remote servers. If the remote database contains IP-based access controls, it will evaluate them using the database link's domain rather than the original client domain.