Best Practices: Security
by Mark Cox, Consulting Engineer and Michael Tiemann, Chief Technical Officer


The first and foremost best security practice is to have a documented security policy. Regardless of how complete or incomplete this policy may be, the policy is the objective reference against which one can measure "what did we say we would do, what did we do, and what do we need to do in the future to do better?" If the policy is found to have had a hole, the policy can be amended and security improved. However, if there is no policy, then every incident becomes a firedrill: can it ever happen again? Will it ever happen again? What if it does happen again? Trying to make those decisions in the absence of a documented security policy leads most often to either spending a lot of money on a solution that doesn't really fix the root problem, or spending nothing, learning nothing, and hoping it doesn't happen again.

Protecting in the absence of threats:

From time to time flaws in software are found. A small subset of these flaws pose security issues, a smaller subset could potentially be exploited by a person with malicious intent, and a smaller subset still actually become exploited and pose an immediate threat. It is best practice for an enterprise to protect against security vulnerabilities even in the absence of known theats, a practice encouraged by the US government in their "National Strategy to Secure Cyberspace" document released in February 2003.

In the last few years there has been huge amounts of publicity over worms; malicious programs designed to automate the exploitation of a software vulnerability and then make use of those exploited machines to spread themselves to others. All of the worms that have affected Linux to date have exploited known vulnerabilities - security flaws that had already been fixed in the software and where patches were available from vendors between two and six months earlier. (see "Open Source Security and the Role of the Vendor") Administrators who kept their machines up to date with the latest security fixes were safe from attack from those worms. But even so, over twenty thousand machines fell victim to a worm called Slapper in September 2002. This particular worm exploited a flaw that was fixed by vendors in July 2002. It is therefore essential for enterprises to put plans and processes in place to deal with applying security updates to their infrastructure.

Keeping systems up to date:

There are a number of best practices that vendors must adopt to help their users keep up to date with security patches. Red Hat has incorporated these practices into our security response policies for dealing with vulnerabilities and reducing the risk to our customers. It is best practice for security updates to:

  • be easy to find
  • be timely
  • be accompanied by clear and accurate information
  • be transparent
  • be easy to reference
  • be easy to install
  • have minimal consequences for upgrading
  • not break existing systems

Vendors must produce stable updates which have minimal consequences and to do this ensure that security updates are released separated from product updates. Vendors such as Red Hat have made it a best practice to backport security fixes when possible. Backporting allows fixes for security flaws to be separated from the rest of a product update, making it easier for customers to apply updates while reducing the risk of the updates having any negative impact on their systems.

It is best practice for vendors to provide automated ways for users to obtain security patches, to help keep machines up to date.

Trusted Information:

Open source gives users transparency over security fixes, making vendors accountable and unable to hide vulnerabilities or deny they exist. Software packaging mechanisms such as RPM enforce a model where users can quickly extract and examine each of the patches being applied for themselves. Vendors that ship the same components can compete with each other on the speed and quality of fixes, keeping them honest. It is a best practice for vendors to give accurate, unbiased information about security flaws so that users can make informed decisions.

Even with these best practices it can sometimes be hard to track down a vulnerability or to research it. Perhaps the press has misunderstood the issue or is caught in an unintentional game of Chinese Whispers. Adding to the confusion are researchers reporting the issue who may be tempted to deliberately exaggerate the seriousness of an issue in order to promote their abilities or the need for their services. With security advisories that have the potential to boost the business of the company making the warning, it is best practice to seek out several sources of information. Better yet, if the source code for the patch is available, read it directly and discuss it with community peers.

As more and more companies began to offer open source software as a commercial offering, the historic method of reporting security incidents became artificial. Before open source became a pervasive commercial solution, each problem found in traditional proprietary software merited its own vulnerability report. Consider the case of UNIX, which fragmented: it was impossible to determine whether commonly-licensed UNIX remained common among different licenses, and whether common or not, each vulnerability could only be serviced by the vendor that owned the code. Thus, even when vulnerabilities were common, each vulnerability needed to be tracked separately. When Linux came onto the scene, vulnerabilities would be found, fixed, and patches widely distributed, almost always before any effective exploitations had been reported. The unintended consequence; however, was that a single patch would show up in a shower of vulnerability reports, and because of the way these metrics were tabulated, Linux appeared far less secure than it was in practice.

Red Hat joined the Mitre editorial board for the Common Vulnerabilities and Exposure Project (CVE) in 2002. The CVE project makes it easier for users to reference and research security vulnerabilities by providing them with a common name. Red Hat has made it a policy to include CVE vulnerability names in all our security advisories back as far as January 2000 and has advocated their use to other vendors and security researchers. Thus, Red Hat users can now track vulnerability reports by canonical incident (providing an apples-to-apples comparison against other software platforms) or by vendor incident (providing apples-to-apples comparison against other vendors).

A NIST recommendation "Use of the CVE Vulnerability Naming Scheme Within its Acquired Products and Information Technology Security Procedures" recommends that government agencies give substantial consideration to buying products and services compatible with the CVE naming scheme and to periodically monitor their systems for vulnerabilities listed in the CVE vulnerability naming scheme.

It is a best practice for vendors to provide CVE references with security updates and in communication with their users, and for enterprises to demand it.

Licensing and Legislation:

You've written your security policy. You've been diligent about protecting your systems, even in the absence of known threats. You've been careful to verify (made easier if you've chosen open source) that a security patch will not break existing systems. You've applied security patches independent from product updates so that you can be sure that you are not creating new security problems as you fix the old. Late one night you discover that your desktop systems can be totally compromised any time they touch the Internet, and especially when they touch the Internet trying to locate new security patches. You learn of a patch that can solve this problem, but to apply the patch, you have to accept the license agreement that comes with it. Such a bargain may sound bizarre, like something from "The Monkey's Paw", but indeed, sometimes life imitates art. Considering that most security policies are designed to defend against 3rd parties disabling system functionality and/or surreptitiously installing code without one's knowledge, how do these licensing terms fit within (or violate) such security policies:

"You agree that in order to protect the integrity of content and software protected by digital rights management ('Secure Content'), the Vendor may provide security related updates to the OS Components that will be automatically downloaded onto your computer. These security related updates may disable your ability to copy and/or play Secure Content and use other software on your computer. If we provide such a security update, we will use reasonable efforts to post notices on a web site explaining the update."

Moreover, legislation such as the DMCA may actually make it illegal to proactively find system vulnerabilities and use scientific methods (such as sharing of knowledge) to address these vulnerabilities. Companies who take security seriously should become sufficiently informed and active to ensure that their security is not compromised for the convenience of others. More information on unintended consequences of the DMCA.

Accepting the reality:

Vendors need to face up to the reality that users don't always apply security updates. Sometimes enterprises can't apply updates because they don't have the time, resources, or energy. Sometimes enterprises can't apply updates because those updates would violate some part of their security policy. Or maybe the system administrator is on holiday, or quit, or the machine has been abandoned.

So knowing that users may still not apply security updates for whatever reason, vendors must therefore make it a best practice to reduce the risk, to reduce the impact of any future security vulnerabilities, to protect users from themselves as well as from malicious attackers. Vendors can do this by careful choice of the software they include, by intelligent use of defaults and permissions, and by making security features such as firewalls a commons. Vendors can also innovate and design technologies to reduce the impact of security flaws. Red Hat designed exec-shield for the Linux kernel to make it much harder for attackers to exploit the majority of security flaws, raising the security bar, reducing the risk of exploitation.

Open Source software puts users in a position of control. With a single-source vendor you are locked in and have to rely on the vendor to give you everything you need. With the source available and in the open, enterprises are empowered by choice, and can not be held to ransom by a sole-supplier.