Chapter 6     Designing the Replication Process

Replication Concepts

When you consider replication, you always start by making the following fundamental decisions:

• What information you want to replicate.
  
  
• Which server (or servers) holds the master copy, or read-write replica, of that information.
  
  
• Which server (or servers) holds the read-only copy, or read-only replica, of the information.
  
  
• What should happen when a read-only replica receives an update request; that is, to which server should it refer the request.

These decisions cannot be made effectively without an understanding of how the Directory Server handles these concepts. For example, when you decide what information you want to replicate, you need to know what is the smallest replication unit that the Directory Server can handle. The following sections contain definitions of concepts used by the Directory Server. This provides a framework for thinking about the global decisions you need to make.

Data Consistency

Consistency refers to how closely the contents of replicated databases match each other at a given point in time. When you set up replication between two servers, part of the configuration is to schedule updates. The supplier server always determines when consumer servers need to be updated and initiates replication.

Directory Server offers the option of keeping replicas always synchronized or of scheduling updates for a particular time of day or day in the week. The advantage of keeping replicas always in sync is obviously that it provides better data consistency. The cost is the network traffic resulting from the frequent update operations. This solution is the best in cases where:

• You have a reliable high-speed connection between servers.
  
  
• The client requests serviced by your directory are mainly search, read, and compare operations, with relatively few update operations.

In cases where you can afford to have looser consistency in data, you can choose the frequency of updates that best suits your needs or lowers the affect on network traffic. This solution is the best in cases where:

• You have unreliable or intermittently available network connections (such as a dial-up connection to synchronize replicas).
  
  
• The client requests serviced by your directory are mainly update operations.
  
  
• You need to reduce the communication costs.

In the case of multi-master replication, the replicas on each supplier are said to be loosely consistent because at any given time, there can be differences in the data stored on each supplier. This is true even when you have selected to always keep replicas in sync for two reasons:

• There is a latency in the propagation of update operations between suppliers.
  
  
• The supplier that serviced the update operation does not wait for the second supplier to validate it before returning an "operation successful" message to the client.

Single-Master Replication

In the most basic replication configuration, a supplier server copies a replica directly to one or more consumer servers. In this configuration, all directory modifications occur on the read-write replica on the supplier server, and the consumer servers contain read-only replicas of the data.

The supplier server must perform all modifications to the read-write replicas stored on the consumer servers. Figure 6-1 shows this simple configuration.

Figure 6-1    Single-Master Replication

The supplier server can replicate a read-write replica to several consumer servers. The total number of consumer servers that a single supplier server can manage depends on the speed of your networks and the total number of entries that are modified on a daily basis. However, you can reasonably expect a supplier server to maintain several consumer servers.

Multi-Master Replication

In a multi-master replication environment, master copies of the same information can exist on multiple servers. This means that data can be updated simultaneously in different locations. The changes that occur on each server are replicated to the others. This means that each server plays both roles of supplier and consumer.

When the same data is modified on multiple servers, there is a conflict resolution procedure to determine which change is kept. The Directory Server considers the valid change to be the most recent one.

Multiple servers can have master copies of the same data, but, within the scope of a single replication agreement, there is only one supplier server and one consumer. That means that to create a multi-master environment between two supplier servers that share responsibility for the same data, you need to create more than one replication agreement. Figure 6-2 shows the configuration for a two-way multi-master replication.

Figure 6-2    Multi-Master Replication Configuration (Two Suppliers)

In the above illustration, supplier A and supplier B each hold a read-write replica of the same data.

To create a multi-master environment between four supplier servers that share responsibility for the same data, you need to create more than four replication agreements. Figure 6-3 and Figure 6-4 illustrate two sample configurations of four-way multi-master replication agreements. Keep in mind that the four suppliers can be configured in different topologies and that there are many parameters that have direct impact on the topology selection.

Figure 6-3 illustrates a fully connected mesh topology where all four supplier servers feed data to the other three supplier servers (and to the consumer servers). There are a total of twelve replication agreements among the four supplier servers. This topology provides high server failure tolerance at the expense of high fan-out for every supplier.

Figure 6-3    Multi-Master Replication Configuration (Four Suppliers)

Figure 6-4 illustrates a topology where each supplier server feeds data to two other supplier servers (and to the consumer servers). Notice that there are only eight replication agreements among the four supplier servers, as opposed to the twelve agreements shown for the topology in Figure 6-3. The topology shown in Figure 6-4 is beneficial in situations where the possibility of two or more servers failing at the same time is negligible. Because each supplier has only two fan-outs, such a configuration is useful in reducing the network traffic and making the servers less busy.

Figure 6-4    Multi-Master Replication Configuration (Four Suppliers)

The total number of supplier servers you can have in any replication environment is limited to four. However, the number of consumer servers that hold the read-only replicas is not limited.  

Note	Directory Server supports four-way multi-master replication; that is, replication topologies comprising four supplier servers.

Figure 6-5 shows the replication traffic in an environment with two suppliers (read-write replicas in the illustration), and two consumers (read-only replicas in the illustration). This figure shows that the consumers can be updated by both suppliers. The supplier servers ensure that the changes do not collide. 

Figure 6-5    Replication Traffic in a Multi-Master Environment

Cascading Replication

In a cascading replication scenario, a hub supplier receives updates from a supplier server and replays those updates on consumer servers. The hub supplier is a hybrid: it holds a read-only replica, like a typical consumer server, and it maintains a change log like a typical supplier server.

Hub suppliers pass the master data on as they receive them from the original suppliers. For the same reason, when a hub supplier receives an update request from a directory client, it refers the client to the supplier server.

Cascading replication is useful, for example, if some network connections between various locations in your organization are better than others. For example, suppose the master copy of your directory data is in Minneapolis, and you have consumer servers in Saint Cloud as well as Duluth. Suppose your network connection between Minneapolis and Saint Cloud is very good, but your network connection between Minneapolis and Duluth is of poor quality. Then, if your network between Saint Cloud and Duluth is of acceptable quality, you can use cascaded replication to move directory data from Minneapolis to Saint Cloud to Duluth.

This cascading replication scenario is illustrated in Figure 6-6.

Figure 6-6    Cascading Replication Scenario

The same scenario is illustrated from a different perspective in Figure 6-7 below. It shows how the replicas are configured on each server (read-write or read-only) and which servers maintain a change log.

Figure 6-7    Replication Traffic and Change logs in Cascading Replication

Mixed Environments

You can combine any of the scenarios outlined in the previous sections to best fit your needs. For example, you could combine a multi-master configuration with a cascading configuration to produce something similar to the scenario illustrated in Figure 6-8.

Figure 6-8    Combined Multi-Master and Cascading Replication

Replication Survey

The type of information you need to gather from your survey to help you define your replication strategy includes:

• Quality of the LANs and WANs connecting different buildings or remote sites and the amount of available bandwidth.
  
  
• Physical location of users, how many users are at each site, and what is their activity.
  
  
• The number of applications that access the directory and relative percentage of read/search/compare operations to write operations.
  
  If your messaging server uses the directory, you need to know how many operations it performs for each email message it handles. Other products that rely on directory are typically products such as authentication applications or meta-directory applications. For each one, you must find out the type and frequency of operations that are performed in the directory.

• The number and size of the entries stored in the directory.
  
  A site that manages human resource databases or financial information is likely to put a heavier load on your directory than a site containing engineering staff that uses the directory for simple telephone book purposes.

Replication Resource Requirements

Using replication requires more resources. Consider the following resource requirements when defining your replication strategy:

• Disk usage -- On supplier servers, the change log is written after each update operation. Supplier servers receiving many update operations may see higher disk usage.
  
  In addition, as there is a single change log on every supplier server, if a supplier contains multiple replicated databases, the change log will be used more frequently, and the disk usage will be even higher.

• Server threads -- Each replication agreement consumes one server thread. So, the number of threads available to client applications is reduced, possibly affecting the server performance for the client applications.
  
  
• File descriptors -- The number of file descriptors available to the server is reduced by the change log (one file descriptor) and each replication agreement (one file descriptor per agreement).

Using Replication for High Availability

Use replication to prevent the loss of a single server from causing your directory to become unavailable. At a minimum, you should replicate the local directory tree to at least one backup server.

Some directory architects argue that you should replicate three times per physical location for maximum data reliability. How much you use replication for fault tolerance is up to you, but you should base this decision on the quality of the hardware and networks used by your directory. Unreliable hardware needs more backup servers.  

Using Replication for Local Availability

Your need to replicate for local availability is determined by the quality of your network as well as the activities of your site. In addition, you should carefully consider the nature of the data contained in your directory and the consequences to your enterprise in the event that the data becomes temporarily unavailable. The more mission-critical the data, the less tolerant you can be of outages caused by poor network connections.

You should use replication for local availability for the following reasons:

• You need a local master copy of the data.
  
  This is an important strategy for large, multinational enterprises that need to maintain directory information of interest only to the employees in a specific country. Having a local master copy of the data is also important to any enterprise where interoffice politics dictate that data be controlled at a divisional or organizational level.

• You are using unreliable or intermittently available network connections.
  
  Intermittent network connections can occur if you are using unreliable WANs, as often occurs in international networks.

• Your networks periodically experience extremely heavy loads that may cause the performance of your directory to be severely reduced.
  
  Among other reasons, performance is affected in enterprises with aging networks, which may experience these conditions during normal business hours.

Using Replication for Load Balancing

Replication can balance the load on your Directory Servers in several ways:

• By spreading your user's search activities across several servers.
  
  
• By dedicating servers to read-only activities (writes occur only on the supplier server).
  
  
• By dedicating special servers to specific tasks, such as supporting mail server activities.

One of the more important reasons to replicate directory data is to balance the workload of your network. When possible, you should move data to servers that can be accessed using a reasonably fast and reliable network connection. The most important considerations are the speed and reliability of the network connection between your server and your directory users.

Directory entries generally average around one Kbyte in size. Therefore, every directory lookup adds about one Kbyte to your network load. If your directory users perform around ten directory lookups per day, then, for every directory user, you will see an increased network load of around 10,000 bytes per day. Given a slow, heavily loaded, or unreliable WAN, you may need to replicate your directory tree to a local server.

You must carefully consider whether the benefit of locally available data is worth the cost of the increased network load because of replication. If you are replicating an entire directory tree to a remote site, for instance, you are potentially adding a large strain on your network in comparison to the traffic caused by your users' directory lookups. This is especially true if your directory tree is changing frequently, yet you have only a few users at the remote site performing a few directory lookups per day.

If your directory tree on average includes in excess of 1,000,000 entries, and it is not unusual for about ten percent of those entries to change every day, then if your average directory entry is only one Kbyte in size, you could increase your network load by 100Mbyte per day. However, if your remote site has only a few employees, say 100, and they are performing an average of ten directory lookups a day, then the network load caused by their directory access is only one Mbyte per day.

Given the difference in loads caused by replication versus that caused by normal directory usage, you may decide that replication for network load-balancing purposes is not desirable. On the other hand, you may find that the benefits of locally available directory data far outweigh any considerations you may have regarding network loads.

A good compromise between making data available to local sites and overloading the network is to use scheduled replication. For more information on data consistency and replication schedules, refer to Data Consistency.

Example of Network Load Balancing

Suppose your enterprise has offices in New York and Los Angeles. Each office has specific subtrees that they manage, shown in the figure.  

Each office contains a high-speed network, but you are using a dial-up connection to network between the two cities. To balance your network load:

• Select one server in each office to be the supplier server for the locally managed data.
  
  Replicate locally managed data from that server to the corresponding supplier server in the remote office.

• Replicate the directory tree on each supplier server (including data supplied from the remote office) to at least one local Directory Server to ensure availability of the directory data. You can use multi-master replication for the suffix managed locally and cascading replication for the suffix that receives a master copy of the data from a remote server.  

Example Replication Strategy for a Small Site

Suppose your entire enterprise is contained within a single building. This building has a very fast (100 MB per second) and lightly used network. The network is very stable, and you are reasonably confident of the reliability of your server hardware and OS platforms. Also, you are sure that a single server's performance will easily handle your site's load.

In this case, you should replicate at least once to ensure availability in the event your primary server is shut down for maintenance or hardware upgrades. Also, set up a DNS round-robin to improve LDAP connection performance in the event that one of your Directory Servers becomes unavailable.

Example Replication Strategy for a Large Site

Suppose your entire enterprise is contained within two buildings. Each building has a very fast (100 MB per second) and lightly used network. The network is very stable and you are reasonably confident of the reliability of your server hardware and OS platforms. Also, you are sure that a single server's performance will easily handle the load placed on a server within each building.

Also assume that you have slow (ISDN) connections between the buildings, and that this connection is very busy during normal business hours.

Your replication strategy follows:

• Choose a single server in one of the two buildings to contain a master copy of your directory data.
  
  This server should be placed in the building that contains the largest number of people responsible for the master copy of the directory data. Call this Building A.

• Replicate at least once within Building A for high availability of directory data.
  
  Use a multi-master replication configuration if you need to ensure write-failover.

• Create two replicas in the second building (Building B).
  
  
• If there is no need for close consistency between the supplier and consumer server, schedule replication so that it occurs only during off peak hours.
  
  

Replication and Access Control

The directory stores ACIs as attributes of entries. This means that the ACI is replicated along with other directory content. This is important because Directory Server evaluates ACIs locally.

For more information about designing access control for your directory, refer to chapter 7, Designing a Secure Directory.

Replication and Directory Server Plug-ins

You can use replication with most of the plug-ins delivered with Directory Server. There are some exceptions and limitations in the case of multi-master replication with the following plug-ins:

• Attribute Uniqueness Plug-in
  
  You cannot use multi-master replication with the attribute uniqueness plug-in at all because this plug-in can validate only attribute values on the same server, not on multiple servers in the multi-master set.

• Referential Integrity Plug-in
  
  You can use the referential integrity plug-in with multi-master replication providing that this plug-in is enabled on just one supplier in the multi-master set. This ensures that referential integrity updates are made on just one of the supplier servers and propagated to the others.  

Replication and Database Links

When you distribute entries using chaining, the server containing the database link points to a remote server that contains the actual data. In this environment, you cannot replicate the database link itself. You can, however, replicate the database that contains the actual data on the remote server.

You must not use the replication process as a backup for database links. You must backup database links manually. For more information about chaining and entry distribution, refer to chapter 5, Designing the Directory Topology.

Figure 6-9    Replicating Chained Databases

Schema Replication

In all replication scenarios, before pushing data to consumer servers, the supplier server checks whether its own version of the schema is in sync with the version of the schema held on consumer servers.

If the schema entries on both supplier and consumers are the same, the replication operation proceeds.

If the version of the schema on the supplier server is more recent than the version stored on the consumer, the supplier server replicates its schema to the consumer before proceeding with the data replication.

If the version of the schema on the supplier server is older than the version stored on the consumer, you will probably witness a lot of errors during replication because the schema on the consumer cannot support the new data.

A consumer might contain replicated data from two suppliers, each with different schema. Whichever supplier was updated last will "win," and its schema will be propagated to the consumer.