OpenShift Security Hardening for the healthcare industry

table { border: #ddd solid 1px; } td, th { padding: 8px; border: #ddd solid 1px; } td p { font-size: 15px !important; }

In all industries, data lies at the heart of an organization, and data needs stringent, consistent and restricted access controls for risk reduction and protection against cybersecurity threats.

It’s important to secure all sensitive data, of course, but extra care must be taken to ensure that personally identifiable information (PII) and personal health information (PHI) are properly protected. At the same time, that information also has to be available for use by authorized people and applications.

There are a number of different strategies healthcare organizations can use to avoid data leaks. Here we present a prioritized list of measures for hardening Red Hat OpenShift using information from the OpenShift Security Guidelines and other sources.

Our prioritization includes these healthcare industry-specific perceived threats:

1. Data leak prevention
2. Cyberattack prevention
3. Insider threat reduction (the “evil admin” problem)
4. Software supply chain attack prevention

There are many other attack vectors, but for this article we’ll be focusing on these four use cases.

1. Data leak prevention

Personal Health Information (PHI) is more valuable on the black market than credit card credentials or regular Personally Identifiable Information (PII), so there’s a higher incentive for cyber criminals to target medical databases so they can sell the PHI or use it for their own personal gain. Data leaks can also severely damage an organization’s reputation and decrease trust with customers and other healthcare partners.

Collecting and interpreting sensitive patient data is critical for healthcare organizations. Organizations must also protect that data against cybersecurity threats while also complying with the Health Insurance Portability and Accountability Act (HIPAA) and other standards to reduce the risk of data leaks.

Embedding security controls throughout the full platform and development cycles helps organizations manage sensitive data and ensure that access is only granted to authorized and authenticated users.

Infrastructure security

You can consider some of these measures to  better secure your infrastructure and prevent data leaks:

• Ensure unencrypted data isn’t exposed through the base operating system by using self-encrypting disks (SED), encrypted enterprise storage or encrypted block storage provided by an external cloud provider.

• Red Hat Enterprise Linux CoreOS also supports Linux Unified Key Setup (LUKS) disk encryption secured with Trusted Platform Module (TPM) v2 or Network Bound Disk Encryption (NBDE), which requires a Tang server. 

• The Federal Information Processing Standard Publication 140-2 (FIPS-140-2) is a standard defining a set of security requirements for the use of cryptographic modules. Red Hat OpenShift Data Foundation is now using FIPS validated cryptographic modules as delivered by Red Hat Enterprise Linux OS/CoreOS (RHCOS).

• The use of a service mesh such as Istio can help provide strong identity, powerful policy, mutual TLS encryption and authentication, authorization and audit (AAA) tools to protect services and data. Istio can also provide certificate management using X.509 certificates.

Tampering can lead to elevation of privileges, information loss and damage. Ensuring that the data is not maliciously altered is critical to maintaining data security.  While some Red Hat OpenShift components such as container images, etcd, APIServer and Kubernetes configuration files can be tampered with, the platform provides protection for these components out of the box:

• TLS is the general tool for protecting against tampering of data in transit.

• All communication to the Red Hat OpenShift APIServer is over TLS.

• Only the Red Hat OpenShift APIServer is allowed to communicate with etcd.

• All Red Hat OpenShift components are managed with Kubernetes operators, ensuring that only supported configuration changes are allowed. If an unsupported configuration change is made, the operator for that component will reset the change or mark the component as degraded. Red Hat OpenShift operators help manage configuration drift.

• The container runtime is protected with Security-Enhanced Linux (SELinux), kernel and network namespaces, Cgroups and, optionally, secure computing mode (seccomp) profiles.  

• Red Hat OpenShift Security Context Constraints (SCCs) give the Red Hat OpenShift administrator the ability to make use of Red Hat Enterprise Linux security features, such as dropping Linux capabilities, and apply those restrictions across pods deployed to the cluster. The restricted SCC is applied to all worker nodes, preventing the running of privileged containers, and preventing access to the host network and filesystem. If it’s necessary to run a privileged pod, a privileged or custom SCC can be applied to the pod.

These additional recommendations will further enhance the security of your Red Hat OpenShift cluster data at rest:

• Restrict access to the control plane by configuring a separate ingress controller

• Restrict access to repositories with Red Hat OpenShift configuration files

• Perform bootstrapping over Secure Shell (SSH) and safeguard the SSH keys

• Restrict access to the image registry and associated repositories

• Encrypt the etcd datastore

• Implement a process to review the security context of the pod manifest file to ensure that unnecessary privilege and access is not requested

• Install and configure the Compliance operator to ensure required compliance controls are implemented

• Install and configure File Integrity Operator to continually run intrusion detection checks on cluster nodes

Red Hat OpenShift implements a principle of least privilege security model, with built-in authentication and authorization. By default, privileged containers cannot run on compute nodes in Red Hat OpenShift, but you still need to be sure that the applications you deploy cannot be spoofed, as this could lead to compromised applications and data.

Although the Red Hat OpenShift security model requires its components to authenticate via mutual TLS (mTLS), it’s only as secure as the certificate authority (CA) that issues the certificates. A compromise there could lead to an ineffective mTLS layer.

Some recommendations to navigate this include:

• Only using internal Red Hat OpenShift CAs within the cluster

• Using custom certificates—one for authentication of internal components and one for external components

• Setting the automountServiceAccountToken to false for pods that don’t need to communicate with the APIServer

• Using secrets to mount certificates into pods that can be used for authentication of pod identity

• Integrating with an external vault to manage application secrets

• Using the configurations for expirationSeconds and audience if a service token needs to be mounted

Application development security

DevSecOps is a complex undertaking, especially as DevSecOps tools grow and change at a fast pace. Containers and Kubernetes add more complexity and open up new attack vectors and security risks. Development and operations teams must make security—including Kubernetes security—an integral part of the application life cycle to safeguard critical IT infrastructure and protect confidential health data.

To reduce the risk of data leakage, we recommend that you use built-in capabilities within the OpenShift platform and Kubernetes to increase container security. Examples include pod security policies, network traffic controls, cluster ingress and egress controls, role-based access controls (RBACs), integrated certificate management, and network microsegmentation.

2. Cyberattack prevention

The U.S. Department of Health and Human Services produces a report that shows security breaches across the U.S. healthcare industry in the last two years. This report shows that almost 73% of security breaches are due to hacking or IT incidents.

To reduce the number of cyberattacks, security should be an integral part of the design, implementation and maintenance of any system containing or processing personal health information.

Infrastructure security

By default, Kubernetes allows pods to talk to each other unimpeded, so network policies should be used as a key security control to prevent an attacker from moving laterally through a container environment. The biggest security gains regarding limiting external attacks come from applying ingress policies, so we recommend focusing on those first, and then adding egress policies. 

Network policies provide a simple way to add filtering. If a pod is matched by selectors in one or more NetworkPolicy objects, the pod will accept only connections that are allowed by at least one of those NetworkPolicy objects. The following are policy options that can be set to control pod network communications:

• Deny all traffic.

• Only allow connections from the OpenShift Container Platform ingress controller.

• Only accept connections from pods within a project.

• Only allow HTTPS traffic based on pod labels.

• Accept connections by using both namespace and pod selectors.

• Block “front end” pods from directly communicating with database pods—middle tier pods should be used for database connections.

For more information on creating ingress policies, please see the Guide to Kubernetes Ingress Network Policies.

Application development security

To prevent delays in application deployment and to fully realize the benefits of containers and Kubernetes, organizations should “shift left” with security. This means security should be integrated into the development process as early as possible so any security challenges can be addressed well before the build and deployment stages.

We recommend that you unify security across your organization using a DevSecOps approach. This extends responsibility for security across teams, rather than having a single, disconnected team responsible for setting security policy.

It is worth noting that attackers are less likely to attack infrastructure and they will probably attempt attacks through the user interface or application programming interface (API). For this reason, additional security hardening techniques should be taken into account such as using TLS everywhere (creating a “Zero Trust Architecture”) and increased access logging to monitor who has accessed sensitive health data. Additional actions such as image vulnerability scanning will also decrease the risk of vulnerabilities within containers.

The ideal solution would be to have mTLS/TLS all the way down from the APIs to the databases.

3. Insider threat reduction (the “evil admin” problem)

According to a Techjury Study, 66% of organizations consider malicious insider attacks or accidental breaches more likely than external cyberattacks. Additionally, the number of insider-originating incidents has increased by 47% over the last two years. 

An example of an “evil admin” within the healthcare industry is that of Jesse William McGraw, aka "GhostExodus". McGraw pleaded guilty in 2010 to computer tampering charges for putting malware on a dozen machines at a hospital in Texas, including a nurses' station that had access to medical records. He was later sentenced to nine years and two months in prison for installing malware on computers.  

Technology alone cannot remove this type of attack, but it can be used to mitigate attempts and to limit the attack surface. Privileged user accounts are required for various legitimate purposes, but they should be managed and monitored to stop either intentional or accidental breaches in security.

Infrastructure security

Identity and Access Management (IAM) and RBAC are core features of Red Hat OpenShift. Below, we describe the controls available for administrators, engineers and developers who are using Red Hat OpenShift to restrict access to platform resources. These control descriptions should not be confused with application-level IAM and RBAC, such as those used with a website or web application.

Direct access to an application that happens to be running on Red Hat OpenShift is a separate topic and is typically handled by the guest application.

RBAC objects in Red Hat OpenShift determine the level of access a user is granted. Cluster administrators can use roles and bindings to control users’ access to resources in Red Hat OpenShift.

Authorization is controlled through the following:

Red Hat OpenShift creates a cluster administrator, kubeadmin, after the installation process completes. This user has the cluster-admin role automatically applied and is treated as the root user for the cluster.

After you define an identity provider and create a new cluster-admin user, you can remove the kubeadmin user to improve cluster security.

If you remove the kubeadmin user before another user is a cluster-admin, then Red Hat OpenShift Container Platform must be reinstalled. It is not possible to undo this action.

Linked to RBAC and access controls, maintaining immutable and non-repudiated audit logs is critical. Organizational administrators and authorised external systems which have agreed and authorized reasons for accessing the logs should have the ability to create alerts using these logs and should be accountable for their use and dissemination.  

These types of controls are beneficial for recording and examining information system activity, especially when determining if a security violation has occurred. When you have the full audit trail, you are able to better make a risk assessment and advise more specific remediation. The U.S. Department of Health and Human Services recommends that organizations:

 “Implement hardware, software, and/or procedural mechanisms that record and examine activity in information systems that contain or use electronic protected health information.”

Application development security

Adding proper authentication to an application may be cumbersome since correctly implementing standards and interoperable authentication flows, like OpenId Connect (OIDC), can be a challenging task, but these are crucial for increasing security and reducing risk.  

For any new application using microservices architecture, it’s possible to delegate all the authentication and authorization concerns to a building block such as Red Hat Single Sign-on. 

Regarding authentication and authorization, it is advised to use OIDC, Open Authentication (OAUTH) or Security Assertion Markup Language (SAML) to protect all resources.

4. Preventing software supply chain attacks

The now famous “SolarWinds Hack,” (which impacted the National Institutes of Health among other government agencies) is one example of how dangerous software supply chain attacks are. They rely on compromising the software Continuous Integration/Continuous Delivery (CI/CD) process by introducing malicious software into regular software builds, preferably software with a wide reach into “interesting” companies.

To defend against this type of attack, a multitude of changes have to be made to internal processes to ensure a non-compromised software build across the whole build chain.

Building security into your applications is critical for cloud-native deployments. Securing your containerized applications requires that you:

• Use trusted container content.

• Use an enterprise container registry.

• Control and automate building containers.

• Integrate security into the application pipeline.

The CI/CD pipeline is at the core of a secure software supply chain and helps  prevent supply chain attacks because developers remove the vulnerabilities before the application goes into production. Adding automation to the process allows IT teams to deliver resources faster, supporting rapid proofs of concept, development, testing and deployment into production.

The bottom line

Protecting sensitive data is important in all industries, but is particularly vital in healthcare due to the nature (and value) of personal health information. In this post, we’ve outlined some strategies towards mitigating four high priority types of security threats that healthcare IT teams can use to ensure their infrastructure and application development processes are as secure as possible.

Learn more

• OpenShift for developers

• Try Red Hat OpenShift

• Red Hat Certified Specialist in OpenShift Administration

• Red Hat OpenShift I: Containers & Kubernetes

• Red Hat OpenShift Administration II: Operating a Production Kubernetes Cluster

• Red Hat OpenShift Administration III: Scaling Kubernetes Deployments in the Enterprise