ProductsDesktop Server For Scientific Computing For IBM POWER For IBM System z For SAP Business Applications Red Hat Network Satellite ManagementExtended Update Support High Availability High Performance Network Load Balancer Resilient Storage Scalable File System Smart Management Extended Lifecycle SupportDeveloper Studio Portfolio Edition Web Framework Kit Application Platform Web Server Data Grid Portal Platform Red Hat JBoss A-MQ Red Hat JBoss Fuse SOA Platform Business Rules Management System (BRMS) Data Services Platform Messaging JBoss Operations Network JBoss Community or JBoss enterprise
SolutionsApplication development Business process management Enterprise application integration Interoperability Operational efficiency Security VirtualizationSolaris to Red Hat Enterprise Linux Migration overview Migrate from your UNIX platform How to migrate to Red Hat Enterprise Linux Upgrade to the latest Red Hat Enterprise Linux release JBoss Enterprise Middleware Benefits of migrating to Red Hat Enterprise Linux Migration services Start a conversation with Red Hat
TrainingPopular and new courses Red Hat JBoss Administration curriculum Core System Administration curriculum JBoss Middleware Development curriculum Advanced System Administration curriculum Linux Development curriculum Cloud Computing, Virtualization, and Storage curriculum
ConsultingBusiness Process Management Cloud and Virtualization Custom Software Development Enterprise Data and Storage Systems Management Migrations
Third-Party Severity Ratings
June 27, 2007
by Security Team
The National Vulnerability Database (NVD) is a US Government repository of vulnerability management data that includes databases of security checklists, security related software flaws and impact metrics. It provides a public severity rating for all the vulnerabilities named by the CVE (Common Vulnerabilities and Exposures), a list of standardized names for vulnerabilities and other security exposures. The ratings can be “Low,” “Medium” or “High”. Each rating is generated automatically based on the CVSS (Common Vulnerability Scoring System) score its analysts calculate for each issue.
At Red Hat, we’ve been interested for some time in seeing how well those map to the severity ratings that Red Hat give to issues. We use the same ratings and methodology that many others in our industry use - we assign “Critical” to things that have the ability to be remotely exploited and we obviously react and fix these vulnerabilities with the highest priority. Our remaining three levels, “Important,” “Moderate” and “Low,” also take into account the potential risk of a flaw and are treated according to this risk.
To explore our vulnerability ratings in relation to NVD, we took the last 12 months of all possible vulnerabilities affecting Red Hat Enterprise Linux 4 - from 126 advisories across all components - from one of our metrics pages and compared them to NVD using its provided XML data files. The result broke down like this:
- Red Hat:
Seems like a good comparison on the surface. But what is implied is that all the issues Red Hat rated as “Critical” were mapped to NVD as “High.” This actually isn’t the case. If you look at the breakdown of what the NVD rated as “High,” “Medium” and “Low,” you’ll see that varied Red Hat ratings are included in each collection, skewing the comparison.
The NVD rated 90 vulnerabilities as “High.” These 90 included the following vulnerability ratings from Red Hat:
- 23 Critical
- 24 Important
- 35 Moderate
- 8 Low
The NVD rated 61 vulnerabilities as “Moderate.” These 61 included the following vulnerability ratings from Red Hat:
- 9 Critical
- 18 Important
- 22 Moderate
- 12 Low
The NVD rated 152 vulnerabilities as “Low.” These 152 included the following vulnerability ratings from Red Hat:
- 7 Critical
- 32 Important
- 62 Moderate
- 51 Low
In summary, NVD rated 90 issues that affected Red Hat as “High.” Out of those, we rated only 47 of those “Critical” or “Important.” So almost half of the issues that the NVD rated as “High” actually only affected Red Hat with “Moderate” or “Low” severity. Likewise, there were some Red-Hat-rated “Critical” and “Important” vulnerabilities under the NVD’s “Low” rating.
This difference in ranking turns out to be less mysterious than one might imagine. It seems that many of the differences are due to the way vulnerabilities affect open source software. Take for example the Apache HTTP server. Lots of companies ship Apache in their products, but all ship different versions with different defaults on different operating systems for different architecture compiled with different compilers using different compiler options. Many Apache vulnerabilities over the years have affected different platforms in significantly different ways. We’ve seen an Apache vulnerability that leads to arbitrary code execution on older FreeBSD, that causes a denial of service on Windows, but that was unexploitable on Linux for example. But this flaw had a single CVE identifier.
Recent competitive studies have attempted to rate Red Hat on our performance based on NVD “High” severity issues. By doing this, these studies are actually including our performance for 43 “Moderate” and “Low” issues – issues that we typically defer to be fixed via future updates. In reality, our policy is to fix the things that are genuinely “Critical” and “Important” the fastest, and we have an impressive record for fixing “Critical” issues.
So it’s no wonder that recent competitive vulnerability studies that use the NVD mapping when analyzing Red Hat vulnerabilities have some significant data errors.
Basically, we feel that you can’t use the generic severity ratings maintained in third-party databases as an accurate assessment of how that issue affects specific products, such as Red Hat Enterprise Linux. For multi-vendor software, the severity rating for a given vulnerability may very well be different for each vendor’s version. This is a level of detail that vulnerability databases such as NVD don’t currently capture. So, comparisons of response times that rely on the accuracy of third-party severity ratings will always be biased against open source software, and is another reason why such comparisons are pretty meaningless.
Red Hat provides open and transparent metrics on its track record correcting security issues here.