Compliance is the topic of this entry in Red Hat’s Security series! In March 2021 the Red Hat Security Ecosystem team began introducing DevOps Security topics to help you learn how Red Hat weaves together DevOps and security to master the force called DevSecOps.  We explain how to assemble Red Hat products and our security ecosystem partners to aid in your journey to master deploying a comprehensive DevSecOps solution.

Compliance can mean a lot of different things, but in this post we will focus on regulatory compliance, and what to think about when attempting to audit and meet compliance requirements within a Red Hat OpenShift deployment. 

Compliance methods and technologies help you adhere to industry and government regulations and corporate policies. They automate compliance validation and reporting throughout DevOps, helping you simplify audits and avoid costly regulatory fines and lawsuits.

However, getting security right is very complicated with endless areas of confidentiality, integrity and availability.  A growing number of industries have legal or contractual obligations towards compliance and have created industry standards. 

Security software vendors have created tools to allow end-users, service providers, and other product builders to more easily evaluate security compliance against the whole framework versus looking at hundreds of different controls. Here’s a sample of some of the common compliance standards we run into:

  • Payment Card Industry Data Security Standard (PCI-DSS) increases controls around cardholder data to reduce credit card fraud.

  • ISO/IEC 27001 is a set of international standards on how to manage information security to help organizations make the information assets they hold more secure.

  • U.S. Health Insurance Portability and Accountability Act (HIPAA) is a United States federal law to protect sensitive health information from being disclosed unless the patient consents. .

  • European Union General Data Protection Regulation (GDPR) harmonizes national data privacy laws in the European Union to protect the privacy of citizens of the EU..

  • Security Technical Implementation Guides (STIGs) are configuration standards consisting of cybersecurity requirements for a specific product used widely in the Federal industry.

  • Federal Risk and Authorization Management Program (FedRAMP) is a United States government program to standardize security assessments, authorization, and continuous monitoring for cloud products and services.

DevSecOps framework The Red Hat DevSecOps Framework (diagram above), includes two compliance methods: compliance audit and technical controls. 

Compliance audit is a function which typically scans a config, container, cluster, or system to report if the object in question is in compliance or not.  In the DevSecOps Framework, “compliance controls” weave automation with technical controls. This can help automate and enable proper actions, or prevent actions that would result in non-compliance.  As an example, a few concrete examples of what items are audited:

  • Configuration of your deployments for production readiness and security, i.e. required labels, and privileged containers.

  • Configuration of your Dockerfiles, i.e. exposed ports, and required Dockerfile lines.

  • Known vulnerabilities affecting the base image and application dependencies.

  • Risk analysis of packages in your images. For example, we assess that including nmap in your images increases risk.

These items are typically audited throughout the life cycle within integrations at build automation, container admission and the running cluster as shown in this table:

Integration Point

Description

Build Automation Compliance Audits

A key integration point to catch non-compliance is at the application build process to save time and money later on in the life cycle. Applications, container images, and configurations should all be checked at this stage. An example compliance check at this stage is to ensure the base image being used is hardened using CIS benchmarks. 

Container Orchestration Compliance Audits

As images are being tested and deployed into running clusters, compliance audit results should be very similar to the results found in the build, however compliance audit technology will now have access to the running cluster and environment, which will produce more contextual results.  For example, audits can now look at port forwarding and oc exec commands into pods. It is important to note, finding compliance issues early in the build is still ideal because this will lessen the amount of rework required during test and deployment.

Container Orchestration Compliance Controls

The technical controls during container orchestration are typically Admission Controls, which can alert on or prevent pod creation as images are being deployed.  For example, blocking privileged containers is a common compliance admission control. 

Cluster Compliance Audits

Ongoing compliance scans are critical for non-static items like vulnerabilities. While your image was free and clean from vulnerabilities at build time, the second after your build, those results become stale. In addition to images, the cluster operating systems, like RHEL CoreOS should be continuously audited for compliance

Cluster Compliance Controls

Typically, the technical controls in a running cluster are of the remediation type whether they are automatic or manual.  It may not be practical to stop a running container in production when a new critical vulnerability is found, but rather provide the appropriate alerting, logs, and any possible mitigation or suggested fixes to an administrator who can escalate an update to the container image.

 

Red Hat OpenShift Hardening Guides can help

In January 2021 Red Hat published the Red Hat OpenShift Container Platform Hardening Guide, This document provides prescriptive guidance for establishing a secure configuration posture for OpenShift 4.5+, and  was inspired by the CIS Kubernetes benchmark v1.6. 

Red Hat is working with CIS to publish a CIS OpenShift benchmark based on this content. The Red Hat Hardening Guide is specifically for OpenShift as it provides audit commands that are specific to the OpenShift architecture As RHEL CoreOS is designed to be managed as part of the cluster, audit commands are written using the OpenShift oc client, including audit commands that check host-level filesystem permissions and ownership.

Recommendations exist for the major Kube components, like the Control and Worker node configs, API server, etcd, and kubelet. The next example shows some of the content found in each recommendation:

Recommendation: Verify that RBAC is enabled

Description: Turn on Role Based Access Control.

Rationale: Role Based Access Control (RBAC) allows fine-grained control over the operations that different entities can perform on different objects in the cluster. It is recommended to use the RBAC authorization mode

Audit: OpenShift is configured at bootstrap time to use RBAC to authorize requests… To verify, run the following commands:

... # For 4.6, verify that the authorization-mode argument includes RBACoc get configmap config -n openshift-kube-apiserver -ojson | jq -r '.data["config.yaml"]' | jq '.apiServerArguments["authorization-mode"]'

For OCP 4.6 and above, verify that the authorization-modeargument includes RBAC.

Remediation: None. It is not possible to disable RBAC.

It is important to note, the majority of the settings in the hardening guide are in place by default in Red Hat OpenShift. The audit information for these settings is provided so that you can verify that the cluster admin has not made changes that would be less secure than the OpenShift defaults. A small number of items require configuration. Finally, there are some recommendations that require decisions by the customer, such as audit log size, retention and related settings.

There are a lot of commands to run and analyze, and while this guide provides the answers in this static document, you will really want to automate these checks and continuously monitor them, which is exactly what Red Hat’s security Partners can do with their solutions.

Top 5 todos to help pass your audit

So, how are you going to make your auditor’s job easy and pass your first audit?  Well, Bill Montgomery and Nathan Kinder, a couple of Red Hat Engineering Senior Managers, have expertise in completing product compliance audits and offer the following todos to get started: 

  1. Documented Policies and Procedures: Documentation is typically greater than 50% of an Audit. Policy documentation is all about what you do for things like access, change control, backups and data retention. Procedure documentation is how you do things, for example, how you scan for vulnerabilities or how you delete data.

  2. Change Control: Ensure that all changes to production are auditable and follow the principle of Separation of Duties to ensure no individual can affect a change to production acting alone.  GitOps is a process used frequently in DevOps that can implement Change Control efficiently and effectively.

  3. Encryption everywhere: All current compliance frameworks require encryption for data at rest (i.e., on disk) and in transit (i.e., network communications). Especially for network encryption, ensure server processes are configured to enforce negotiation of strong security protocols (TLS >= 1.2, HSTS).

  4. Vulnerability management: Ensure your development process is scanning for software vulnerabilities early and often.  For containers, vulnerabilities can exist in all layers of the stack, including the operating system, the cluster, the base layer, any application services added to run your application like web servers or 3rd party dependencies, and the custom code that makes up the application itself.

  5. Security Monitoring: FedRAMP calls this set of practices “ConMon” or “Continuous Monitoring”, which require some frequency and breadth/depth of security monitoring.   Ensure to aggregate security data from things like audit logs, file integrity scans and vulnerability scans and send them to some type of SIEM. Use software to analyze security events and alert incident response to anomalous audit events.

Getting started on these five items can help go a long way to helping you pass your audit, and impress your auditor. 

However, be sure to follow all the specific framework requirements for the compliance framework you are trying to achieve.  Hopefully, we provided some insight on Compliance for DevSecOps, with a few key things to know when integrating compliance audits and technical controls in your software life cycle.  For more information, please visit www.red.ht/DevSecOps or email us at DevSecOpsHelp@redhat.com


About the author

Dave Meurer currently serves as a Principal Solution Architect on the Red Hat Global Partner Security ISV team, where he owns technical relationships and evangelism with security independent software vendor partners of Red Hat. Before joining Red Hat, he spent nine years in the Application Security industry with Synopsys and Black Duck, where he served in similar roles as the director of technical alliances and sales engineering.

Read full bio