Red Hat Enterprise Linux 6 Kernel: An Overview and Genealogy

When Red Hat announces a new major Red Hat Enterprise Linux release, such as with the recent Beta availability of Red Hat Enterprise Linux 6, invariably among the first questions asked is, “What is the kernel version number?” The answer to this question is never a simple one-number reply. The construction of an enterprise-caliber kernel is an extremely complex exercise that requires close evaluation of hundreds of individual features and interactions. This blog outlines how we create our Red Hat Enterprise Linux kernels.

Because Red Hat is the leading contributor to the Linux kernel, we always have a large number of development initiatives in progress. Some initiatives span many years in a continuum of enhancements - a key example being the realtime latency enhancements that we have been developing over many kernel versions. So a complete feature may be delivered in stages over several releases, or perhaps delivered all at once when development is finally completed. Additionally, in any upstream kernel release, there will be some features that are production-ready, and others that are still a work in progress. The key balance we endeavor to strike when constructing an enterprise-focused kernel is to:

• Include mature enterprise features
• Include latent enablers in anticipation of future advances, such as scalability accommodations for hardware systems, that may not be delivered for several years
• Disable components that are not enterprise relevant, or not mature enough to be considered production-ready for Red Hat Enterprise Linux deployments

We believe that Red Hat’s leadership in kernel development and our large customer and partner community makes us uniquely positioned to sort out which features are ready for production support from Red Hat and which are not. Among many evaluation factors, we closely review our practical testing experience of various upstream kernel features. In fact, there are cases where we are not the first to productize our own development initiatives - when it comes to enterprise deployments, doing it right, not first, is our primary goal.

Taking into consideration all the above factors, the Red Hat Enterprise Linux 6 kernel is not a straightforward inheritance of a single upstream kernel version.

At the time of this blog posting, the latest upstream kernel version that is close to completion is 2.6.34. The Red Hat Enterprise Linux 6 kernel includes numerous subsystems and enhancements from 2.6.34, as well as its predecessor versions. As a result, the Red Hat Enterprise Linux 6 kernel cannot be simply labeled as any particular upstream version. Rather, the Red Hat Enterprise Linux 6 kernel is a hybrid of the latest several kernel versions. And, as Red Hat provides regular updates over the lifecycle of the product, we expect that the Red Hat Enterprise Linux 6 kernel will incorporate selected features from future upstream kernels that have yet to be developed.

Another important part of the construction of the Red Hat Enterprise Linux 6 kernel is a commitment to preserve ABI (application binary interface) compatibility, on two distinct levels.

• Firstly, at the system call level: the interface used by libraries and applications – which is crucial in preserving application compatibility and the Red Hat Enterprise Linux ISV ecosystem.
• Secondly, at the kernel ABI level: used by third-party device drivers and kernel code.

To help hold these ABIs consistent for the full life of the product, Red Hat Enterprise Linux implements structural modifications. These allow for the later incorporation of features that would otherwise have been precluded, without perturbing the existing ABI.

This process is not new. Red Hat has been issuing enhancements to Red Hat Enterprise Linux during the Production I phase of the lifecycle. For example, Red Hat Enterprise Linux 4, released in 2005, included new hardware support through 2009 and continues to be supported through 2012. Red Hat Enterprise Linux 5 launched in March 2007, will continue to receive hardware enhancements through 2011 and will be supported until 2014. Of course, within a given release’s maintenance stream, stability is the key acceptance criteria when we consider adding new kernel features. So in cases where we consider a new feature to be too risky or invasive to introduce into a maintenance update, it becomes one of the factors that triggers the introduction of a new major kernel version, such as the kernel in the upcoming Red Hat Enterprise Linux 6 release.

The following list gives a taste of just a few of the many kernel enhancements that were not practical to deliver in the Red Hat Enterprise Linux 5 maintenance stream, so are under consideration for Red Hat Enterprise Linux 6:

• Cgroups (Control Groups). Cgroups provide a fine-grained means of setting policy for process resource consumption, covering IO, memory, and CPU utilization. This is particularly useful to establish control policies on virtualized guests, providing a means of enforcing per-guest SLAs.
• Realtime determinism enhancements. The benefits of many of the advances in timer precision and runtime predictability that were first introduced in the Red Hat Enterprise MRG Realtime product are now expected to be provided as standard capabilities in Red Hat Enterprise Linux 6. These are particularly useful for time-sensitive environments, such as those supported by trading and high-speed messaging applications. Red Hat Enterprise MRG will also continue to be provided as a standalone product, offering additional determinism capabilities.
• Power management. Of several features that improve power management, perhaps the most important is called the “tickless” kernel. This reduces the traditional requirement for the hardware clock to interrupt the operating system (typically 1,000 times per second) in order to maintain accurate system and event timing. When processing requirements allow, the clock interrupt rate can be reduced dramatically, thereby allowing processors to enter low power states when idle.
• Virtualization. Red Hat Enterprise Linux 6 provides virtualized guests with numerous I/O and memory optimizations, and enhanced support for virtualized hardware devices. These allow guests to directly access I/O devices, bypassing hypervisor overhead. This support also includes improved access to large memory pages, which are particularly useful for enabling high-performance virtualized database deployments. This continues our initiative to seamlessly allow all workloads to benefit from the flexibility provided by virtualization while minimizing overhead.
• KSM (Kernel Shared Memory). This feature allows condensing of duplicate pages into a single page. It is particularly useful among virtualized guests, which may, for example, be running the same operating system, and so have many identical memory pages. It is useful for Windows guests as they initialize their full range of memory with zeros at system startup, thereby needlessly consuming unused memory.
• KMS (Kernel Mode Setting). The Red Hat Enterprise Linux kernel is used in both server and client environments, so enhancements to the graphics subsystem are of great importance. The new KMS feature provides numerous performance optimizations, as well as improving the appearance of system startup so, for example, it cleanly transitions to the login screen. KMS also enhances the ability to support multiple monitors, projectors, docking stations and suspend/resume.
• Scalability. A number of kernel enhancements that are under consideration allow Red Hat Enterprise Linux 6 to optimally support the latest generation of large memory and multi-CPU configurations - and to have headroom to grow for many years to come.

This list highlights just a few of the many new features that we expect to debut in the Red Hat Enterprise Linux 6 kernel.

In addition to the kernel work completed by Red Hat engineers, we also work hand-in-hand with engineers from our close partners. There is a continuum of features that we continue to mature for appearance in forthcoming maintenance releases, such as 6.1, 6.2 etc., as well as bigger features targeted for Red Hat Enterprise Linux 7.

The Red Hat Enterprise Linux 6 Beta is available for public download here.