Issue #17 March 2006

Risk Report: A year of Red Hat Enterprise Linux 4

1. Introduction

Red Hat® Enterprise Linux® 4 was released on February 15th 2005. This report looks at the state of security for the first year from release, including metrics, key vulnerabilities, and the most common ways users were affected by security issues. We will show some best practices that could have been used to minimize the impact of the issues and also take a look at how Red Hat security innovations like Exec-Shield and SELinux fared.

We've called this article a "Risk Report"; we see risk as being a function of the security vulnerabilities that have affected users of Enterprise Linux 4 in the first year (vulnerabilities), and how users were compromised through exploits and worms (threats).

All data in this report is for Red Hat Enterprise Linux 4 AS from release day, February 15th 2005 through February 14th 2006 unless otherwise stated.

2. Vulnerabilities

At first sight it may appear that Red Hat released a lot of updates over the year [1] , publishing a total of 183 security advisories to address 422 vulnerabilities. This is a worst-case statistic covering all vulnerabilities regardless of their risk, and only applies to a system that had installed every available package, which is not a default or likely installation.

With the release of Enterprise Linux 4 we started publishing severity levels to help users figure out which advisories were the ones that mattered the most. Providing a prioritized risk assessment helps customers to understand and better schedule upgrades to their systems, being able to make an more informed decision on the risk that each issue places on their unique environment.

Cut down on the number of alerts you see. Register your systems with the Red Hat Network to get customized notifications for security updates for the packages your systems have installed. If you want to see all security updates for every version and package subscribe to the enterprise-watch-list mailing list as well

Red Hat rates the impact of individual vulnerabilities on a four point scale [2] designed to be an at-a-glance guide to how worried Red Hat is about each security issue. The scale takes into account the potential risk of a flaw based on a technical analysis of the exact flaw and it's type, but not the current threat level. Therefore the rating given to an issue will not change if an exploit or worm is later released for a flaw, or if one is available before release of a fix.

Table 1. Severity Rating

To monitor the number of vulnerabilities customers need to deal with we use two metrics: the number of critical vulnerabilities, and severity weighted number of vulnerabilities a day, the vulnerability workload index.

This vulnerability workload index gives a measure of the number of important vulnerabilities that security operations staff would be required to address each day. The higher the number, the greater the workload and the greater the general risk represented by the vulnerabilities. The workload index is calculated in a similar way to the workload index from NIST [3].

For a given month, Vulnerability workload = ((number of critical and important severity vulnerabilities published within the last month) + (number of moderate severity vulnerabilities published within the last month/5) + (number of low severity vulnerabilities published within the last month/20)) / (days in the month)

        A graph showing the workload index decrease from an initial
        high of 1.6 to around 0.4 average over the year

Figure 1. Vulnerability Workload for Enterprise Linux 4 AS full install

The initial peak during the first month was expected, as the code for Enterprise Linux 4 had frozen a few months prior to release, leading to a backlog of issues needing to be fixed shortly after release.

Table 2. Number of vulnerabilities for each severity

When Red Hat Enterprise Linux is installed, the user gets a choice [4] of installing either the default selection of packages, or making a custom selection.

Table 2 shows that a default install of Enterprise Linux 4 AS [5] would not be vulnerable to any critical flaws, and this is because most of the critical flaws have been in web browsers or plug-ins. Firefox and Mozilla are not installed by default on server systems. Client systems (Enterprise Linux WS and Red Hat Desktop) do include Firefox, Mozilla, and Helixplayer by default, leading to 14 default critical vulnerabilities. A non-default installation of AS, selecting every available package, would yield a system with the maximum possible number of critical vulnerabilities in the year, 19. [6]

2.1. Critical Flaws

Vulnerabilities that are critical severity are the ones that can pose the most risk to an organisation. By definition a critical vulnerability is one that could potentially be exploited remotely and automatically by a worm. We stretch the definition to also include those flaws that affect web browsers or plug-ins where a user only needs to visit a malicious web site in order to be exploited.

Over the past year a number of research reports have rated the response times of vendors with a metric called "days of risk" which measures the number of days it takes for a vendor to produce a patch for a vulnerability after the patch is first disclosed to the public. This is a useful and interesting metric, although researchers have said that a more useful metric would be to include the time from notification to the vendor so that the results cannot be influenced by vendors that have policies to keep vulnerabilities private until such time as they release a regularly scheduled update.

In the table below we've included the date the vulnerability was first known to the public and the date an update for Enterprise Linux 4 was available.

Table 3. Critical flaws in Enterprise Linux 4

As shown in the table, fixes for 100% of the critical flaws were available from Red Hat Network within two calendar days of public disclosure of the vulnerability. 73% of the critical flaws were fixed within one calendar day. This fast response time is essential to reduce the risk from these critical vulnerabilities.

Also included in the table is the date that Red Hat was notified about this issue: we were given some advance notice for 13 of these 19 vulnerabilities (although in some cases this pre-notification was only a day or two in advance of public disclosure). Because the software we ship is open source we're likely not to be the only company shipping the vulnerable application and therefore Red Hat is not in sole control over the date the issue is made public. This can lead to much shorter response times between issues being reported and being made public. This situation means we can't try to artificially reduce our "days of risk" statistics by tactics such as holding off disclosure of issues for a long period until a patch is ready.

2.2. Riskiest packages

It's always seemed like we have to produce a security advisory for some packages (like ethereal) every month; so we analyzed the statistics for Enterprise Linux 4 to find out which packages we responsible for the most vulnerabilities. In order to take into account the severity we've weighted the vulnerabilities

(To rate them I used a metric of "Critical + Important/5 + Moderate/25 + Low/100")

Table 4. Top 10 packages with the worst Enterprise Linux 4 security history

These top 10 packages listed actually counted for 70% of all weighted vulnerabilities.

From the list above, Red Hat Enterprise Linux 4 AS only has the kernel, cups, and gpdf packages as part of a default installation.

Although ethereal did have a total of 52 vulnerabilities through the year, they were all rated moderate or low severity and therefore the overall weighted risk is much lower, keeping it out of the top 10.

An easy way to reduce the number of security alerts that apply to your system is to remove packages that you don't need. Pay particular attention to the packages which have been riskiest to date. When installing a new system, carefully choosing the right Enterprise Linux variant and package set to match the task can cut down on the number of alerts.

3. Threats

So far, we've looked at the theoretical vulnerabilities affecting the platform. But to get a more accurate picture of the risk we also need to take into account the actual threat. This section looks at exploits and worms written for flaws in software packages we ship.

3.1. Exploits

There is evidence that shows that many exploits for security vulnerabilities are kept private and never released to the public. However a number of these exploits leak publicly or are released publicly on purpose. We wanted to find out how often this happens and get a list of the available exploits for vulnerabilities that affected the first year of Enterprise Linux 4. This can help determine the level of threat.

For the purposes of this study we are interested in those exploits that have the potential to cause lasting damage and so we ignore any exploits that are designed to create a denial of service (crash). We did, however, include exploits which are labelled "proof of concept" where the published exploit may only cause a crash - these often show techniques that a skilled attacker could turn into a full exploit.

For the year we found a total of 28 exploits for software we ship, all affecting 32-bit Intel architectures. Slightly under half of the exploits are for buffer overflows vulnerabilities either on the stack or the heap and in most cases the Exec-Shield technologies should help prevent remote exploits of these vulnerabilities due to randomization and enforcement of a non-executable stack.

3.1.1. Kernel exploits

Most vulnerabilities in the Linux kernel lead to one of two consequences: a local unprivileged user can cause the machine to crash, or a local user can elevate their privileges (and gain root privileges). We found exploits for five vulnerabilities that had the potential to allow an unprivileged user to get root on an unpatched Enterprise Linux 4 system. Of the five, one required the system to be using bluetooth drivers (CVE-2005-0750), and another was exploitable only on systems with more than one CPU (CVE-2005-0001). The remainder (CVE-2005-0736, CVE-2004-1235, and CVE-2005-0531) would work on any unpatched system. In some of the cases the public exploit would need slight adjustment to work on a vulnerable Red Hat Enterprise Linux 4 kernel.

3.1.2. Browser exploits

Over a third of the public exploits we found were for flaws in web applications; all but one of which were targeted at the Mozilla suite (Mozilla, Firefox, Thunderbird). For each exploit, any code execution would be limited to being run with the same rights as the user that is running the vulnerable browser. It is therefore best practice to never use a web browser or graphical email client as root. Table 5 describes the browser exploits.

Table 5. Exploits for browser flaws

3.1.3. Other user-complicit exploits

Table 6 lists the exploits we discovered that require some user involvement.

Table 6. Exploits for user-complicit flaws

3.1.4. Servers and services exploits

Finally, Table 7 lists the exploits we discovered that affect server applications and services.

Table 7. Exploits for flaws in servers and services

The way to reduce your risk from exploits is to make sure your systems have all applicable security updates installed. The Red Hat Network can help with this.

3.2. Worms

Worms take advantage of vulnerabilities in packages in order to compromise systems, then use the compromised system to seek out other systems to infect. By our severity definition, any vulnerability that could be exploited in this way would be classed as critical. Earlier we listed every vulnerability rated as critical severity. In reality though, only a subset of those vulnerabilities could be actually used by worms since we also class as critical some browser vulnerabilities where a victim has to take action (for example visiting a malicious web page).

Linux worms have been quite scarce in the last few years, and the anti-virus vendors who track malware only recorded two during the period.

  • Linux/MARE was discovered in November 2005 and was a worm that spread by exploiting a flaw in PHP-Nuke. PHP-Nuke is not shipped as part of Red Hat Enterprise Linux.

  • Linux/Lupper was discovered in December 2005 and was a worm designed to exploit a flaw, CVE-2005-1921, in the PHP PEAR XML-RPC server package through a number of third party PHP scripts. None of the affected third party applications were shipped as part of of Red Hat Enterprise Linux. Additionally, a PHP update in July 2005 fixed the underlying vulnerability and hence protected against this worm.

The vulnerability exploited by the Lupper worm was an application flaw rather than the more usual buffer overflow or format string flaw. This meant that technologies designed to protect against overflows such as Exec-Shield had no preventative effect.

The flaw allowed a remote attacker to send a string that would then be interpreted by a PHP exec() command. The worm therefore triggered this flaw and send a string similar to shown here:

exec("cd /tmp;wget;chmod u+x exe;./exe"); 

In this case, the SELinux targeted policy which is installed on Enterprise Linux 4 by default does prevent running of the newly downloaded binary from the /tmp directory; therefore preventing the Lupper worm from executing it's malicious code and spreading. However the policy doesn't prevent, for example, arbitrary Perl scripts from being run, so it would have been possible for a modified version of the worm would have been able to work around the particular default SELinux policy we shipped.

4. Mitigation

Red Hat is continually developing technologies to help reduce the risk of security vulnerabilities, and a number of these were consolidated into Red Hat Enterprise Linux 4. The biggest of these being SELinux and Exec-Shield. Exec-Shield is a project which encompasses support for the No eXecute (NX) memory permission, simulating NX via segment limits, Position Independent Executables (PIE), gcc, and glibc hardening. Part of the glibc hardening included adding checks to prevent the exploitation of a "double-free", an obscure programming flaw that has led to some high-profile exploits of services such as wuftpd in the past.

In August 2004, the MIT team announced a double-free security flaw in the MIT Kerberos 5 Key Distribution Center program. [7] For Red Hat Enterprise Linux 4 users we were able to downgrade the severity of this flaw from critical to important due to the glibc hardening preventing the exploitation of this double-free flaw.

5. Conclusion

The aim of this report was to look at the risk that users of Red Hat Enterprise Linux 4 would have encountered in the first year of release. We've shown that although on the surface it looks like Red Hat releases a large number of security advisories that in fact many of them do not apply to usual or default installations, and only a subset are high severity.

  • A customized installation selecting all packages would have been vulnerable to 19 critical security issues in the year, although 100% had updates available from the Red Hat Network within two calendar days of them being known to the public

  • A default installation of Enterprise Linux 4 AS was not vulnerable to any critical security issues

  • We found 28 public exploits that could have affected a customized full installation, although attempts to exploit many of them would have been caught by standard Enterprise Linux 4 technologies

  • The most likely successful exploits would have allowed a local unprivileged user to gain root privileges

  • Two worms targeting Linux systems were discovered, but both affected third party PHP applications not shipped in Red Hat Enterprise Linux 4

It would be foolish to draw conclusions about the state of security in Red Hat Enterprise Linux 4 solely on the basis of this report, however what we've tried to do is to show the level of vulnerability and threat and hence overall platform risk for the past.

The policies of the Red Hat Security Response team are designed to reduce the risk of security vulnerabilities:

  • An emphasis on providing the fastest possible turnaround for critical vulnerabilities

  • Releasing updates for important security issues as soon as possible rather than batching them into monthly or quarterly updates

  • Transparency in our metrics and providing more details of vulnerabilities in linked bug tracking database

Most of the raw data used to generate the statistics in this report along with some tools to analyze it are available on-line [8] from the Red Hat Security Response Team and updated regularly.

[5] There are four variants of Red Hat Enterprise Linux 4; two targeted at server solutions with Enterprise Linux AS and ES, and two targeted at client solutions with Enterprise Linux WS and Red Hat Desktop. The package set available in Enterprise Linux WS and Red Hat Desktop is a subset of that available in Enterprise Linux AS.

[6] For the purposes of these statistics we look at the worst case, a version of Red Hat Enterprise Linux 4 obtained on the day of release. During the first year two Update releases were made (Update 1 on June 8th 2005 and Update 2 on October 5th 2005). The Update releases are similar to a service pack and contain a roll-up of all security advisories to date. Therefore a user installing Update 1 or Update 2 would only be vulnerable to a subset of the issues.

About the author

Mark Cox manages the Red Hat Security Response Team. Over the last 11 years, Mark has developed software and worked on the security teams of some of the most popular open source projects including Apache, mod_ssl, and OpenSSL. Mark is a founding member of the Apache Software Foundation and the OpenSSL project, and a board member of the Mitre Common Vulnerabilities and Exposure project. In his spare time he hunts the Scottish countryside for small plastic boxes with the aid of a GPS receiver.