You call that DevSecOps? Why your DevSecOps practice may be falling short

Putting the "Sec" in DevOps

Many organizations have been adopting security in some areas of DevOps.

Mary: “Hey, I heard you shifted left last week… all secure now?”

Bob: “Yup, we embraced DevSecOps and love it! Got a huge bonus and told the SOC they could take the rest of the quarter off with full pay.”

If only it were that easy, right?

DevSecOps is not just about shifting left. It is about integrating security throughout the entire DevOps life cycle. We weave security in, end-to-end using multiple methods to ensure there are appropriate monitoring and checks all along the way. Thus, when it gets to production, we’ve minimized the attack surface just in case bad actors penetrate the parameter, or the interior. This is easy to say, but not so easy to do.

So now you’ve decided you want to walk the walk…cool. Let's discuss how. It's all about technology! Right? Nope.

Culture first

If everyone is not rowing in the same direction, the boat will go in circles. So, it's critical to foster a culture of collaboration between developers, operations and, just as important, security teams. The first step is understanding your organizational structure and identifying where things are working well, and people are already collaborating… and where they are not. Use the areas where they work as success stories to help bring people onboard in less collaborative areas of the business.

The key is to help everyone understand why and how security should be involved in the entire DevOps life cycle. It's important security is top-of-mind in every aspect. Including things like: how source control is managed, the choice of base images for developing containers, how to protect data in development as well as production...and much more. Help them understand collaboration will benefit not only catching security flaws in development but also create ways for more effective security response. Better and more visible security early in the cycle should show benefits in operations with reduced number of incidents and quicker remediation.

For instance, operations teams (NOCs) don't always think of an issue being a potential security breach. For them, they look for misconfigurations in software or hardware, or infrastructure problems. Security teams (SOCs) immediately look for a breach. Thus, both sides need to work together to analyze both potentials at the same time since one issue may have caused, or allowed for, the other.

Now let's talk about your secure coding practices. “What? Coding is technology not culture! Right?” - Nope.

Early in the effort is a good time to evaluate and update your coding practices for DevSecOps. While you can’t expect your developers to be security experts, they should be trained in secure coding practices — even if it incurs a time and cost investment. Plus, the developers should want to code securely and believe it is important. Establishing and adhering to coding standards helps keep practices consistent and help developers write clean code. All of this is a huge part of culture.

I bet you're thinking, “I get it. Can we talk about technology now?”

No, not yet.

Process, Process, Process

Did I mention process? Oh yeah…it’s important too. Not long ago it was common to develop an application and then worry about how it was to be deployed, operated and maintained. Application deployments and updates were infrequent because of the pain incurred with long testing cycles, and potential downtime. DevOps changed everything. It sped up development, increased and improved deployments, and removed many operational headaches. Nirvana, right?

Not so fast…

Embracing DevOps meant that we reduced those long testing cycles (which helped reduce bugs and security flaws). We also greatly increased the rate of updates which increases the potential for bugs and security flaws. Additionally, the potential for having multiple versions of an application running at the same time, increased the complexity of solving problems.

To see your vision through you’ll need to either create a single DevSecOps team or pull together a virtual team with multiple disciplines. In both cases, the teams must carry out multiple, different tasks in some combination while using several different tools. Imagine if you had a large environment with a dozen tools that create your cohesive DevSecOps solution. You can understand how workflow standardization, documentation, and automation is essential. Designing agreed-upon processes and executing them will improve efficiency and should improve security throughout the life cycle.

OK, let's talk about the technology...

The art and science of "weaving" a fabric called DevSecOps

Once you’ve nailed the culture part (I have faith in you!) the technology becomes important. Bet you never would have guessed that!

Look at the platforms, tools, and processes you're using for application development, deployment and operations. Your goal is to turn that into a single cohesive system called DevSecOps. Are those tools suited properly for a single cohesive system so your organization can run its business optimally? Nothing will be turn-key so you must craft it yourself -- though I strongly suggest getting help from experts.

Thus, you need to provide a good answer for, “How do we weave our tools and processes together to create an optimal solution for my environment?” Weaving security into the life cycle end-to-end is easy to imagine, but how do we make it reality?

This section itself could be several pages of discussion so we’ll talk briefly about two key items. Determining necessary tool functionality and automation needs to be done together in an iterative feedback loop so they are both optimized for the process you designed previously.

No matter if you are starting fresh or must re-use what you already have…determine the capabilities of the tools you need at every stage of the life cycle. Examine whether the features and functions of those tools adequately solve the problems in your specific environment. Not all tools that solve similar problems actually solve them for all environments in the same way, or optimally. Find the gaps and document them and ensure everyone knows they are there and then work to address them soon as you can. That may mean getting more tools, or more automation, or more people.

Here’s an example of a framework which outlines a basic life cycle and the security methods that could be used in each area. From this, you can pick and choose what you need for your environment and fill in what you have where.

Notice anything missing in that diagram? I hope you do! I’ve mentioned tool and process automation many times but it's not there, right? That’s because it goes in different spots which are dependent on the tools you choose and what you need to automate. It's common to think of automation merely as a way to do more things, quicker, and with less people. But automation can be much more than that.

This is where the feedback and optimization loop really begins. It's critical to determine if and how the needed tools work together. If there is a major problem in answering how to integrate and/or automate a given tool into the system. You need to go back to the toolset and reassess.

Do your tools and technology work together natively or do they require third party or custom built (often in-house) tools? Automate as much as possible. Scanning, compliance, config management, policy enforcement, etc. Anything that is a consistent, repeatable process that doesn't require human intervention to decide on -- thus, not a simple logic tree. When you automate systems, you can inherently improve processes and reduce human errors. Human errors can lead to issues or security gaps.

How the parts weave together matters most…Ecosystem!

Does everything we discussed in the previous sections sound easy? If yes, then congrats, you’re an expert - or you play one on TV. You can stop reading at this point and go get-er-done!

If that prior comment doesn't describe you…please keep reading.

For most organizations going it alone is daunting and potentially dangerous. Selecting tools and vendors that work together closely and will support your DevSecOps design as a single cohesive unit is important. The last thing you want is to design something the vendors say “We’ve never done that” or “We’ve never seen that type of use." Thus, your DevSecOps is still disparate parts you’ve glued, not woven, together.

Ecosystem matters. It's not just about a certification on a platform - it's about how all the players work together and support each other. Ask yourself, are your solutions a hub and spoke design where your organization is the hub linking all the parts to yourself? Thus, you own all those connection points and those points only talk through you!? Or do you use a vendor like Red Hat where the security ecosystem is a mesh, and all the vendors work with each other as well as with Red Hat so your organization is not left holding the bag if an issue arises? Or even not an issue, what about updates to core components like cloud platform, container orgaistrator, key vendor packages, etc.?

With this in mind, you’ll see how Red Hat focuses on our partner ecosystem for container life cycle security and DevSecOps. This diagram gives you a high-level view of how we view the ecosystem:

To start the introduction, we invite you to ourDevSecOps webinars and workshops. Red Hat, along with CyberArk, Palo Alto, Synopsys, and Sysdig present Red Hat’s DevSecOps framework and key security technologies to integrate into your environment.