| LVS Cluster Configuration HOWTO | ||
|---|---|---|
| Prev | ||
This section describes step by step how to create a cluster of two LVS routers and three Web/FTP servers. First, collect information and set up the five systems as explained in the next section. Then, implement the example either from a shell (explained in the section called Implementing the Example from the Shell) or by starting the GUI configuration tool (explained in the section called Implementing the Example with Piranha).
Figure 3 shows the network that will exist after you've set up the LVS routers and real servers. All network addresses shown are for purposes of illustration only.
Figure 3. Layout of the Example Network
|-------|------------------------------------------|---------| Public network
|eth0=1.2.3.2 |eth0=1.2.3.3
|eth0:1=1.2.3.1 (vs1) |
------|----- -------|-----
| active | | backup |
| router | | router |
| | | |
------|----- -------|-----
|eth1=192.168.1.1 |eth1=192.168.1.2
|eth1:1=192.168.1.254 (NAT router) |
|-------|-|------------------|-----------------|---|----------| Private network
|eth0=192.168.1.3 |eth0=192.168.1.4 |eth0=192.168.1.5
| | |
|---------| |---------| |---------|
| rs1 | | rs2 | | rs3 |
|_________| |_________| |_________|
|
From the webmaster, obtain a virtual server IP address. In our example this will be 1.2.3.1. Requests for service at the LVS cluster will be addressed to a fully-qualified domain name associated with this address.
Locate 5 servers and designate their roles: 1 primary LVS router, 1 backup LVS router, 3 real servers. The LVS routers must be Linux boxes running Red Hat 6.1 or later. The real servers may be any platform running any operating system and Web server.
On each LVS router, install two ethernet adapter cards, eth0 and eth1. Create a public IP interface on eth0 and a private IP interface on eth1. The public interface device (eth0) is the heartbeat device. The virtual server address is aliased to this device.
| Primary node | Backup node | |
|---|---|---|
| eth0 | 1.2.3.2 | 1.2.3.3 |
| eth1 | 192.168.1.1 | 192.168.1.2 |
Designate an IP address (192.168.1.254) for the router device (eth1) connecting the active LVS router to the private network. This floating IP address will be aliased to the router device as eth1:1, and will be the gateway to the private network and the default route used by each real server to communicate with the active router.
On each LVS router:
Enable packet forwarding. To do this at system boot, make sure the file /etc/sysconf/network contains the line FORWARD_IPV4=yes. To enable packet forwarding without rebooting, as root issue this command:
echo "1" > /proc/sys/net/ipv4/ip_forward
Enable packet defragmenting. To do this at system boot, make sure the file /etc/sysconf/network contains the line DEFRAG_IPV4=yes. To enable packet defragmenting without rebooting, as root issue this command:
echo "1" > /proc/sys/net/ipv4/ip_always_defrag
Masquerade the private network. Issue this command and put it in /etc/rc.d/rc.local:
ipchains -A forward -j MASQ -s 192.168.1.0/24 -d 0.0.0.0
Decide whether to use the rsh or ssh families for synchronizing LVS cluster files. Verify that your choice is installed such that the LVS routers can log in to one another as root without administrator intervention (see Table 2). In this example, we will choose rsh.
On each LVS router, verify that the LVS cluster software is installed (see the section called Installing the Software.
On each real server, install an ethernet network card, eth0, create an IP address on the same private subnet as in Step 3, and assign a weight to each server indicating its processing capacity relative to that of the others. In this example, rs1 has twice the capacity (two processors) of rs2 and rs3.
| rs1 | rs2 | rs3 | |
|---|---|---|---|
| eth0 | 192.168.1.3 | 192.168.1.4 | 192.168.1.5 |
| weight | 2000 | 1000 | 1000 |
On each real server, verify that the address named in Step 4 (192.168.1.254) is its default route for communicating with the active LVS router.
Decide which program (uptime, ruptime, rup) will be used by the active router to monitor the workload on the real servers. If you choose uptime, each LVS router must be able to connect with each real server without administrator intervention, using the tool family you selected in Step 6. See Table 2 for general enablement instructions. If the selected tool cannot be enabled (for example, one of the real servers is an NT box), the scheduling algorithms that use dynamic load information will still work but the user-assigned weights will be statically applied rather than dynamically adjusted based on load.
Verify that each real server runs an installed and configured httpd server. Note that the real servers must listen on the same port (80 in the example) as the corresponding virtual server.
Verify (for example, using telnet or ping) that each real server can reach hosts on the public LAN. If a real server on the private network cannot reach a host on your LAN, this probably indicates a communication failure between the server and the active router. See Steps 8 and 9.
Determine the runtime parameters. For some of these, you may need to experiment over time to obtain optimal values. In this example, we will use the values listed.
| Value | Parameter Description |
|---|---|
| 1050 | Number of the heartbeat listening port on the primary and backup routers. |
| 2 | Number of seconds between heartbeats. |
| 10 | Number of seconds to wait for a non-responding router to respond before initiating failover. |
| 10 | Number of seconds to wait for a non-responding real server to respond before removing it from the routing table. |
| 180 | When a real server that has been removed from the routing table starts responding again, wait this number of seconds before re-adding the server to the routing table. |
| wlc | Use the Weighted least-connections load-balancing algorithm (assign more jobs to servers that are least busy relative to their load-adjusted weight). See Table 1 for a description of the choices. |
| http | Application. The alternative is ftp. |
| 80 | Virtual server port number. The listening port selected for the virtual server is used on the real servers as well. |
Now we are ready to implement the example. You can do this from a shell as explained in the next section. Or you can use the GUI configuration tool as explained in the section called Implementing the Example with Piranha.
With your favorite editor, open /etc/lvs.cf and set the values shown below. The number on the right is a link to the step in the section called Preliminary Setup that discusses this setting.
# Global section
primary = 1.2.3.2 3
backup = 1.2.3.3 3
keepalive = 2 13
deadtime = 10 13
heartbeat_port = 1050 13
rsh_command = rsh 6
network = nat
nat_router = 192.168.1.254 eth1:1 4
# Per-virtual-server section
virtual server vs1 { 8
address = 1.2.3.1 1
active = 1
load_monitor = ruptime 10
timeout = 10 13
reentry = 180 13
port = 80 13
scheduler = wlc 13
# Per-real-server section
server rs1 { 8
address = 192.168.1.3 8
active = 1
weight = 2000 8
}
server rs2 { 8
address = 192.168.1.4 8
active = 1
weight = 1000 8
}
server rs3 { 8
address = 192.168.1.5 8
active = 1
weight = 1000 8
}
}
Copy the edited configuration file to the backup router.
On the primary router, start the pulse daemon with this command:
/etc/rc.d/init.d/pulse start
Start pulse on the backup router.
On the primary LVS router, as root, type piranha & to start the GUI configuration tool.
Click the Global Settings tab and enter the values shown:
Click the Redundancy tab and enter the values shown:
Click the Virtual Servers tab. On the tab, click Add, then Edit, and enter the values shown:
Click the Real Servers tab. On the tab, click Add, then Edit, and enter the values shown:
Repeat Step 5 for rs2 (192.168.1.4) and rs3 (192.168.1.5). For them, enter a weight of 1000, which will indicate that rs1 has twice the computing capacity of rs2 and rs3.
Click Close to return to the Real Servers tab. There, select each real server and click Activate to activate it.
Click Close to return to the Virtual Servers tab. There, select the vs1 virtual server and click Activate to activate it.
Click the Controls/Monitoring tab and:
Click the Start button (name changes to Stop).
If you check "Add pulse daemon to this runlevel" the cluster will be started at boot time and, on startup of piranha, the button above will usually be labeled Stop.
Log in to the backup router and start pulse with this command:
/etc/rc.d/init.d/pulse start
If you enter the values exactly as shown above, piranha generates at the end of the configuration file the structures shown below for this virtual server and its real servers.
network = nat
nat_router = 192.168.1.254 eth1:1
virtual vs1 {
address = 1.2.3.1 eth0:1
active = 1
scheduler = wlc
load_monitor = ruptime
timeout = 5
server rs1 {
address = 192.168.1.3
active = 1
weight = 2000
}
server rs2 {
address = 192.168.1.4
active = 1
weight = 1000
}
server rs3 {
address = 192.168.1.5
active = 1
weight = 1000
}
}