Operators over easy: an introduction to Kubernetes Operators

You've probably been hearing a lot about Kubernetes Operators, but if you don't work directly with Red Hat OpenShift or another Kubernetes distribution you may not know precisely what an Operator is. In this post, we'll explain what Operators are and why they're important.

To better understand the “what” and the “how” about Kubernetes Operators, we need to understand the problem(s) that motivated the need for Kubernetes Operators.

Kubernetes is notorious in its ability to integrate and facilitate declarative configuration and automation. This was out-of-the-box manageable for most stateless applications. However, for stateful applications this was a bit problematic. How do you manage and persist the state of your application and it’s dependencies? How do you keep the rest of your application going when you add or remove dependencies? Of course, much of this management was done manually and/or required additional personnel resources to help manage (i.e., DevOps) and, in general, required more of your attention.

Much of these pains, boiled down to one ultimate question at hand: How do you effectively automate stateful applications on Kubernetes?

That answer came in the form of what we call Kubernetes Operators.

What’s an Operator?

The "textbook" definition of Kubernetes Operators is "a method of packaging, deploying, and managing a Kubernetes application" and "an application-specific controller that extends the functionality of the Kubernetes API to create, configure, and manage instances of complex applications on behalf of a Kubernetes user."

I also like the explanation given in a Github discussion on Operators vs. controller pattern. “An operator is a Kubernetes controller that understands 2 domains: Kubernetes and something else. By combining knowledge of both domains, it can automate tasks that usually require a human operator that understands both domains.” 

An operator allows you to encapsulate Kubernetes application resources (i.e., Pods, Deployments, Daemonsets, Statefulsets, Jobs, Services, Configmaps, etc.) by creating a CRD (Custom Resource Definition) and a Kubernetes Controller. It manages the state of your application resources and drives deployment and domain specific operations (operational knowledge) of your CRD deployment instance(s).

Operator = Resource + Controller + Domain Specific Operations.

Before we try to provide a short answer to how Operators work, let’s break down each one of these components.

Kubernetes: Resources + Controllers

In Kubernetes, for simplicity’s sake, the basic building blocks that we use to orchestrate our containers can be categorized into either “resources” or “controllers.”

Resources can be composed of things like Nodes, Pods, ConfigMaps, and Services to name a few. Controllers can be composed of things like Deployments, ReplicaSets, and DaemonSets. If you’ve worked with Kubernetes, then you may already be familiar with some of the constructs that we’re looking at.

You are not limited to the readily available resources and controller types offered by Kubernetes. That’s because Kubernetes is a platform. It was made to be expanded upon. And when you treat it like a platform, you can do some pretty amazing things.

You can create your own resources and controllers to communicate and extend the Kubernetes orchestration platform. It’s supported by rich APIs and libraries written in Golang. Consequently, you can leverage those same libraries and APIs to aid you in the creation of your own Custom Resources and Controllers.

By being able to leverage existing resources and creating your own resources and controllers, you can orchestrate and automate your workloads in many ways. It’s not a static platform. Therefore, you’re not boxed into a default set of resources and controllers.

Domain Specific Knowledge/Operations

Domain or Application specific knowledge is another crucial piece within the Operator Pattern. It is generally part of your business logic. This knowledge ties in heavily to a Custom Resource Controller. The operational knowledge that you want to codify and automate gets put into the controller. The Custom Resource itself merely provides useful information that the Resource controller can leverage in the form of custom resource specification and status attributes.

Examples of domain specific tasks and operations may include, but are not limited to:

• Configuration requirements.

• Installation requirements.

• Logging/Security requirements.

• HA/Scaling requirements.

• Application start-up and shutdown routines.

• Process and workflow triggers.

• Application Deployment.

Regardless of what the operation is, what business requirements or needs you must fulfill, these things must be coded within a Custom Resource controller.

Conventional versus Operator based Deployments

The advantages of an Operator can be seen when you start thinking about how you manage your applications deployed into a Kubernetes cluster. Typically, you manage the deployment of each individual resource, which usually constitute the entirety of the application you deploy.

With Operators, these resources are packaged up, and generally expose domain-specific attributes (i.e., specifications) for your deployment from a single point of configuration – your “Operator” instance.

In Figure 2, the real work to allocate and configure the service, deployment, and configmap does not disappear when building an operator. They are managed by your custom resource’s controller. You must specify the resource artifacts and the details of those artifacts you want within your Operator’s CRD instance. You are responsible for the resources you want in your Operator.

You see the immediate benefits of an Operator when you begin configuring and deploying instances of your operator instead of deployments for various resources (i.e., a conventional deployment). Notice how much simpler and less error-prone the operator configuration is in comparison to a conventional deployment.

An Operator-based deployment helps buy consistency by offering you a way to templatize your conventional deployment as well as allow you to develop domain-specific operations (operational knowledge) into your Operator via a controller implementation. Essentially, you create what the community calls a Kubernetes Application.

How do Operators Work?

From a high level, an Operator is composed of the following components:

• A Custom Resource.

• A Resource Controller.

• Codified domain specific knowledge within the Resource Controller.

A CRD allows you to extend the Kubernetes API. A CRD needs a Controller to act upon its presence (i.e., CRD instance).Without a controller for your custom resource, then it’s just a stateless object within Kubernetes.

So then, how does a controller work? What does it do? What’s inside of a controller?

The Resource Controller helps ensure that the current state of a resource matches the desired state of a resource.

An example of this is if we specified the number of pods of a resource through an arbitrary specification/attribute, size, on the custom resource instance. If we increase/decrease the value, then we are setting the desired state.

The following idempotent cycle takes place when a resource change event occurs:

• Observe/watch: In this phase, the controller observes the state of the cluster. Typically this is initiated by observing the events on the custom resource instance. These events are usually subscribed from the custom resource controller. Consider this to be similar to a pub/sub mechanism between the resource controller and cluster.

• Analyze: In this phase, the resource controller compares the current state of the resource instance to the desired state. The desired state is typically reflective of what is specified in the spec attributes of the resource.

• Act/Reconcile: In this phase, the resource controller performs all necessary actions to make the current resource state match the desired state. This is called reconciliation, and is typically where operational knowledge is implemented (i.e., business/domain logic).

The reconciliation cycle generally runs until the desired state is achieved or until an error is thrown. It is not uncommon to see the reconciliation cycle run repeatedly as it treats each run as an idempotent process.

Types of Operators

With regards to the Operator Framework, the following Operator Types are available:

• Golang Operators.

• Helm Operators.

• Ansible Operators.

But wait, there’s more! Outside of the Operator Framework:

• Java Operators.

• Python Operators.

• Javascript/Typescript Operators.

• JSONNET Operators.

Conclusion

Within Red Hat Openshift, Operators provide automation at each level of the stack—from managing the parts that make up the platform all the way to applications that are provided as a managed service. As powerful and accessible as they are, understanding Operators and how to build them can seem complex at first, but is not out of reach as they can be customized, built, and deployed to your needs, with the right set of resources. 

To learn more about how to build your own Operators, check out the Operator SDK as a good starting point. To learn more about Operators in Red Hat’s ecosystem, see Operators on Red Hat OpenShift.