Have you ever done something that was difficult for you to do, but you did it anyway because you cared about the people it would affect? Maybe it was something people honestly forgot you were even doing because you have been doing it for so long? This week I would like to pause and say “Thank You” to the men and women of the SELinux community for watching over us!

Red Hat OpenShift has been using Linux process-to-file type enforcement with multi-category security in its container orchestration platform for 8 years. SELinux has been set to enforcing  in OpenShift since 2011. Red Hat Openshift Online is a publicly accessible hosted service that thousands of developers log into everyday to launch code as containers. Red Hat OpenShift Online had SELinux turned on from the beginning. How about the version of OpenShift you are running inside of your datacenter? That’s right: Red Hat OpenShift Container Platform has had SELinux turned on by default. And we don’t just mean it’s turned on; we mean it is configured to protect you out of the box against real world threats.

I’m afraid I don’t know of another Kubernetes-based container orchestration platform that has used this protection method for this long. Unlike other Kubernetes distributions, Red Hat has bridged the gap between Linux and the container orchestration platform on top, enabling Red Hat OpenShift to track and address security issues across the stack, not just in one layer. And we’re able to do this by default, from day one.

What Happened This Week?

The security vulnerability CVE-2019-5736 was announced this week. Although there are other operating systems and other open source projects that perform type and category enforcement on containers, it is uncommon to have all the pieces configured with SELinux enabled out of the box. It’s even more uncommon to have this configuration extend all the way up to a container orchestration solution that spans your public cloud investments. OpenShift defaults to this configuration on Red Hat Enterprise Linux without you needing to know it is even there. We aren’t just talking about turning setenforce to 1 on a laptop. We are talking about OpenShift type and category enforcing your container usage as a tenant on a single Kubernetes cluster across hundreds of nodes that might be shared with thousands of other tenants.

Pause for a moment and think about what the SELinux type and MCS configuration would look like after a few years of usage in a global company that has given everyone a login to OpenShift. Now imagine giving the world login credentials to your OpenShift cluster like we do at openshift.com. SELinux often does not get credit for all it does in the OpenShift solution. Red Hat’s Linux engineering team’s dedication to security allows you to take advantage of this Red Hat OpenShift configuration in such a highly automated manner. When you think the operating system doesn’t matter anymore, ask your Kubernetes provider if you were protected from CVE-2019-5736 before this week. OpenShift has had that protection since its inception. Try out OpenShift today.

Background

SELinux

One of the most powerful features that Red Hat brings to the table in terms of default security in OpenShift is Security-Enhanced Linux (SELinux). Security-Enhanced Linux (SELinux) is a Linux kernel security module that provides a mechanism for supporting access control security policies. As a conceptual overview it is primarily a labeling system that assigns a label (name) to every process and operating system object. This allows every aspect of kernel operations to be first labeled, second classified, and then ultimately enforced by a set of rules that the provider maintains.

Policy rules control access between labeled processes and labeled objects. The Kernel then enforces the rules that the user has defined in the policies. By default, any rules not allowed are automatically denied, similar to how a firewall will deny any access not explicitly permitted. The images below describe some simple use cases that help visualize the process.

Imagine a system where we define types on objects like cats and dogs. A cat and dog are process types:

*all cartoons by Máirín Duffy

We have a class of objects that they want to interact with which we call food. And I want to add types to the food, cat_food and dog_food.

As a policy writer, we would define that a dog has permission to eat dog_chow food and a cat has permission to eat cat_chow food. In SELinux we would write this rule in policy.

allow cat cat_chow:food eat;

allow dog dog_chow:food eat;

With these rules the kernel would allow the cat process to eat food labeled cat_chow and the dog to eat food labeled dog_chow.

But in a SELinux system everything is denied by default. This means that if the dog process tried to eat the cat_chow, the kernel would prevent it.

This is called Type Enforcement and is essential in protecting the host system from container processes. The container processes can only read/execute /usr files and can only write to the container files. This alone protects the host from the containers, but does not protect containers from other containers, since they are all labeled with the same “type.” In order to provide protection from between containers we will need to enact MCS Enforcement.

MCS enforcement

Although MCS enforcement is not directly related to what protected OpenShift from CVE-2019-5736, it is a good topic to know about in order to round out a complete understanding of how OpenShift uses SELinux. MCS (Multi category security) labeling from a user and system administrator standpoint is straightforward. It consists of configuring a set of categories, which are simply text labels, such as "Fido" or "Spot," and then assigning users to those categories. The system administrator first configures the categories, then assigns users to them as required. The users can then use the labels as they see fit.  This is very helpful to extend the standard SELinux labels via MCS to be applied for container management. The following analogy expands on the example above.

Here we add another section of the label, which we can apply to the dog process and to the dog_chow food. Now we label the dog process as dog:random1 (Fido) and dog:random2 (Spot).

We label the dog chow as dog_chow:random1 (Fido) and dog_chow:random2(Spot).

MCS rules say that if the type enforcement rules are OK and the random MCS labels match exactly, then the access is allowed, if not it is denied.  

Fido (dog:random1) trying to eat cat_chow:food is denied by type enforcement.

Conclusion

The default SELinux protection found in OpenShift, as described above, is a perfect example of defense in depth.  OpenShift, like many other Kubernetes based platforms, has SCC/PSP policies that will prohibit containers from running as the root user.  Stopping root run containers from starting on the platform will also protect from CVE-2019-5736.  In OpenShift, we default to blocking root owned containers, but the setting is configurable.  In the cases people do allow root run containers, the default SELinux configuration in OpenShift will also protect you from CVE-2019-5736.  This is an extra layer of defense that really pays off in this particular case.  You will find OpenShift has many of these layers. Find out more in the 10 Layers of Container Security whitepaper.    

For further information about the CVE-2019-5736 exploit including the RHEL container runtime patch information, please refer to the Red Hat Security Vulnerability Response document found here.