Precision Time Synchronization & Network Latency

In the second installment of “what's new” for networking features in Red Hat Enterprise Linux 7, I'd like to shift the focus and spend some time talking about improvements to network time synchronization and enhancements in the network stack designed to reduce latency.  Note that the previous installment of this two part series was posted during the Red Hat Enterprise Linux 7 beta; now that Red Hat Enterprise Linux 7 is generally available I encourage customers to visit the Red Hat Customer Portal, download this latest release, and to let me know what you think.

Highly Accurate Time Synchronization

Precise time synchronization with microsecond or nanosecond accuracy is increasingly critical to the success of many high-speed, low-latency applications. Whether tracking time on trading transactions or market feeds, improving the time stamp accuracy on archived data, or administration-free time synchronization of servers within a rack (or around the world) for automation or regulatory purposes, precision time synchronization is a key enabler for delivering better performance.

New to Red Hat Enterprise Linux 7,

Chrony is a different implementation of the Network Time Protocol (NTP) that is able to synchronize the system clock faster and with better accuracy than ntpd. Chrony also provides much better response to rapid changes in the clock frequency, which is useful for virtual machines with unstable clocks or power-saving technologies that don't keep the clock frequency constant.

In addition to NTP improvements, Red Hat Enterprise Linux 7 also includes support for the IEEE 1588 Version 2 specification, Precision Time Protocol (PTP). While similar to Network Time Protocol (NTP), the main advantage of PTP is the hardware support found in various network interface cards (NIC) and network switches. When used in conjunction with hardware support, PTP is capable of sub-microsecond accuracy, which is far better than is normally obtainable with NTP. And, by using a GPS-based time source, PTP can even be used to synchronize disparate networks with a high-degree of accuracy.

Performance Optimizations

Introduced nearly 40 years ago,Transmission Control Protocol, or TCP, was designed to facilitate reliable communication between hosts. While networks now operate at much higher speeds and have substantially grown in size since TCP was first introduced, it is still one of the most widely used protocols today for host and application communication.

Red Hat Enterprise Linux 7 has incorporated a number of TCP performance optimizations aimed at reducing overall communication latency for improved application response times, including:

• TCP Fast Open: an experimental TCP extension designed to reduce the overhead when establishing a TCP connection by eliminating one round time trip (RTT) from certain kinds of TCP conversations. It's useful for accelerating HTTP connection handshaking and could result in speed improvements of between 4% and 41% in the page load times on popular web sites.
• TCP Tail Loss Probe (TLP): an experimental algorithm that improves the efficiency of how the TCP networking stack deals with lost packets at the end of a TCP transaction.  For short transactions, TLP should be able to reduce transmission timeouts by 15% and shorten HTTP response times by an average of 6%.
• TCP Early Retransmit: allows the transport to use fast retransmits to recover segment losses that would otherwise require a lengthy retransmission timeout. In other words, connections recover from lost packets faster, which improves overall latency.
• TCP Proportional Rate Reduction (PRR): an experimental algorithm designed to adapt transmission rates to the rates that can be processed by the recipient and by the routers along the way; especially after throttling the rate to prevent an imminent overload. It is designed to return to the maximum transfer rate faster and can help reduce HTTP response times by as much as 3-10%.

Low Latency Sockets using Busy Poll

Even though the networking stack on Linux is considered to be one of the fastest and most robust available today, there are a number of applications where ultra low latency is required. This includes high-frequency trading systems where even a one millisecond advantage can be worth $100 million a year to a major brokerage firm. For many users, the answer until now (while not ideal) has been to use non-standard approaches designed to bypass the network stack from a purely user-space implementation.

Low Latency Sockets is a software implementation designed to reduce networking latency and jitter within the kernel by driving the receive from user context without the need to completely circumvent the Linux networking stack. The native protocol stack is enhanced with a low latency path in conjunction with packet classification by the NIC.

This feature allows an application to enable polling for new packets directly in the device driver allowing packets to quickly find their way into the network stack. This change causes the driver to check the interface for new packets and flush them into the network stack if they exist without causing blocking.

It is designed to be transparent to make polling easy to use by applications and benefits applications sensitive to unpredictable latency by using a top down busy-wait polling method that replaces interrupts for incoming packets.

Mission Accomplished

Now that Red Hat Enterprise Linux 7 has been generally available for just over two months – what’s on your mind?  Have you experimented with any of the new networking features and functionality?  Are there additional areas that you would like to see covered in future posts?  Use the comments section below to share your thoughts.  With any luck you’ll hear from me again soon!