Applying DevSecOps practices to Kubernetes: software supply chain

We’ve discussed ways for you to build a DevSecOps culture and start introducing DevSecOps practices to your development workflows. This post expands on this topic and explores how you could apply the core practices of DevSecOps for your Kubernetes environments. 

DevOps users: the new security users

As part of the shift to a DevSecOps approach, teams must clearly delineate roles and responsibilities when it comes to achieving organization-wide security goals. Security teams must account for DevOps-driven initiatives as part of their overall strategies and can no longer unilaterally block them.

Since DevOps teams will likely take on significant responsibility for operationalizing security tools and practices, they should also be recognized as a new class of security users that is critical to ensuring security is delivered at the speed and scale at which it needs to operate. In turn, security teams should turn their attention to holistically setting security policies and defining best practices that can be easily and repeatedly adhered to by DevOps users.

Kubernetes, in particular, creates the potential for DevOps end users to greatly impact security and enable organizations to shift toward DevSecOps rapidly. This is because a number of native security controls in Kubernetes can be primarily configured and operated directly by these users rather than relying on separate security tools that are managed by security operators. 

These include network policies (used to apply ingress and egress restrictions on pod-to-pod traffic), role-based access control (used to define user and service account roles and privileges), and Open Policy Agent Gatekeeper and security contexts (used to set access and permission constraints on pods that must be met to run within Kubernetes clusters). 

This presents a significant opportunity for security teams to work with these end users to improve the organization’s security posture, more efficiently, for their Kubernetes infrastructure and applications.

Kubernetes security is only as good as where you start: the software supply chain

Successfully integrating DevSecOps practices in Kubernetes environments starts with focusing on how applications are built and making security a part of the process as early as possible, also known as shifting security left. Container images are the standard delivery format for cloud-native and Kubernetes applications and so the initial area for getting DevOps teams involved in security should be the software supply chain.

Traditionally, applications would be subject to updates, patches, and manual changes while running in production. In cloud-native environments, however, developers and DevOps teams rebuild and redeploy containers while treating running containers as immutable. This means that the software supply chain acts as a centralized place for an organization to apply all production changes, whether it means building applications from trusted sources including base operating system images or unit testing, or continuous integration. 

By securing the software supply chain, containerized applications can be better protected prior to being deployed into production environments. Since the software supply chain makes up the primary area where DevOps methodologies are typically applied—e.g., using continuous integration/continuous delivery (CI/CD) pipelines—it is also one of the easiest areas to start extending DevOps practices to DevSecOps practices. 

Security teams can use this to their advantage when protecting cloud-native applications. As an example, one of the most valuable steps that organizations can take to improve their security hygiene for Kubernetes environments is to avoid vulnerabilities in their applications from being introduced into production environments. 

DevOps teams can help accomplish this by incorporating container image scanning into CI/CD pipelines to identify operating system and language-specific vulnerabilities in their application images and set criteria for failing builds or restricting the deployment of pods into production clusters. 

Other security functions that a DevSecOps approach can significantly improve across the software supply chain include: 

• Removing unnecessary tools and components in images to ensure they are only as minimal as needed in order for applications to run. 

• Ensuring secrets are not embedded in images.

• Using immutable tags (sometimes supported by the image registry) to track specific versions of images that are used to deploy containerized applications.

Declarative configuration: an advantage of DevOps for Kubernetes security

Organizations looking to adopt DevSecOps can also look to extend existing DevOps-centric workflows and concepts to security as well. The majority of DevOps teams subscribe to an “Infrastructure as Code” (IaC) model that is based on interfacing with declarative APIs to ensure configurations are specified once upfront and implemented consistently across environments, a so-called “configure once, run everywhere” model. 

This is typically used for provisioning cloud infrastructure, configuring cloud services, setting up recurring API interactions or jobs, and a number of other workflows. 

Extending “Infrastructure as Code” to security functions as part of a DevSecOps initiative can enable a “Security as Code” approach that allows for proactive, automated, and repeatable configuration of security controls across infrastructure and applications.

For example, most production Kubernetes environments are made up of multiple clusters that may or may not be provisioned across multi-cloud or hybrid cloud environments. Organizations must figure out how to securely configure these clusters consistently both at the time their infrastructure is provisioned and when applications are deployed.

Significant differences exist between Kubernetes platforms, especially when considering their default security settings (or the lack thereof)—some expose weaknesses by including and deploying the Kubernetes dashboard, some only support older versions of Kubernetes that contain known vulnerabilities, and others have weak access control (authentication and authorization) and network traffic restrictions. A DevSecOps approach can enable consistent security to compensate for these differences.

The ability to declaratively configure Kubernetes infrastructure settings as well as application constraints (by using a pod specification or deployment manifest) with a DevSecOps model allows organizations to set a security baseline across key aspects of their Kubernetes clusters, regardless of the specific Kubernetes platform and the underlying compute infrastructure environment. 

For example, users can specify a single configuration, such as a Kubernetes network policy, to be propagated across all pods in a deployment rather than having to configure system-level controls on every host in a cluster. This baseline then enables security operators to more easily identify unexpected, anomalous, and malicious activity that warrants further investigation and analysis, streamlining the entire security life cycle.

Declarative configuration also helps eliminate operator error that can often result in misconfigurations that can be exploited. It can also help reduce the overall complexity associated with ongoing configuration management.

Standardization: align teams with a single, familiar operational framework

DevOps practices often aim to promote the notion of standardization as a way to accelerate collaboration and interaction between various stakeholders. DevSecOps extends this by looking for opportunities to streamline workflows and consolidate tooling for security functions. 

This can be achieved by minimizing the number of new interfaces, configurations, and resource models that users have to learn to implement preventative security measures. Instead, aligning security with familiar, open, standardized abstractions and tooling already used by DevOps teams also reduces complexity and avoids potential security issues from arising.

Applied to Kubernetes, a DevSecOps approach has a rich set of standardized abstractions to leverage. To build and ship containerized applications, developers and operators already utilize standards such as image formats and YAML manifests. 

Additionally, Kubernetes has a robust object and resource model that includes concepts such as deployments, replicasets, daemonsets, and others. Many of these are the primary way that pods, the smallest provisionable unit of computing in Kubernetes, are deployed and scaled.

Since developers and operations engineers already utilize these Kubernetes abstractions, security should align with these as well, rather than implementing a custom framework of its own. This means basing security functions such as visibility, configuration management, and runtime detection on existing Kubernetes objects and resources to allow all stakeholders to have a single, consistent understanding of security issues. This enables organizations to realize greater cross-team alignment and simplify the Kubernetes learning curve for teams, thereby saving time and costs when executing their cloud-native security strategies.

Conclusion

In today’s DevOps-driven software environments, security does not fall just within the purview of a centralized security team. Security is everyone’s responsibility. Cloud-native technologies, which encompass containers, microservices, immutable infrastructure, and standardized APIs, are the fundamental building blocks driving accelerated software release cycles in combination with the rise of DevOps, CI/CD automation, and agile methodologies. These technologies are driving software innovations throughout the application development life cycle. They also create new challenges when it comes to security.

In part two of this blog series, we will explore how you can implement DevSecOps principles to improve Kubernetes security analysis and remediation across the full development life cycle. Learn more about how Kubernetes can support a DevSecOps culture in the DevSecOps in Kubernetes whitepaper.