Using Ansible Automation Webhooks for GitOps

If your organization has adopted a DevOps culture, you're probably practicing some form of Infrastructure-as-Code (IaC) to manage and provision servers, storage, applications, and networking. IaC uses human-readable, and machine-consumable, definitions and automation tools such as Ansible. It's also likely that Git is an essential part of your DevOps toolchain, not only for the development of applications and services but also for managing your infrastructure definitions.

With the release of Red Hat Ansible Tower v3.6, part of the Red Hat Ansible Automation Platform, we introduced Automation Webhooks to natively enable easier and more intuitive Git-centric workflows. Such workflows might be found with GitOps-oriented environments.

GitOps is a prescriptive style of Infrastructure as Code based on the experience and wisdom of what works in deploying and managing large, sophisticated, distributed, and cloud-native systems. While you can implement Git-centric workflows, where you treat infrastructure like it is code, that doesn't mean it's GitOps.

In the first half of this post, we'll explore how you can make use of Automation Webhooks. In the second half, we'll cover how to apply these features to creating GitOps pipelines using the unique benefits Ansible provides.

First, let's look at IaC for some context as to how this feature benefits your organization.

Understanding Infrastructures-as-Code

Infrastructure-as-Code (IaC) means managing and provisioning computing resources through human-readable and machine-consumable definition files, rather than using physical hardware configuration or interactive configuration tools. It is considered one of the essential practices of DevOps, along with continuous integration (CI), continuous delivery (CD), observability, and others. DevOps automates development and operations processes and creates a faster release cycle with higher quality. Treating infrastructure like it is code and subjecting it to the same workflows, IaC provides the measurable benefits of cost reduction, faster time to market, and remediation of the risk of costly errors and security vulnerabilities.

Ansible is a prime example of a tool that enables organizations to apply IaC techniques to their DevOps workflow. You're probably here reading this blog post because you are using Ansible and using IaC techniques, whether you realize it or not.

As an Ansible user, you define the desired state of your infrastructure — servers, cloud resources, networking — by using playbooks and roles that are expressed in YAML. This state, along with inventory and variables, is parsed by the Ansible engine and mapped to powerful abstractions (modules) that do the work of reconciling the present state to the declared desired state, whatever that may entail. Need to roll out a new service or adjust your existing infrastructure? Make a change to the Ansible content, commit it, and apply it. Ansible only does what it has to, even if that means doing nothing because your infrastructure is already in the desired state.

Assembling an IaC Ansible pipeline

If you are treating your infrastructure as code, then you need a source control system. These days, when it comes to source control, we're talking about Git, the distributed source control system that has become the de facto standard in the space.

With Git for source control and Ansible to manage the state of your infrastructure, then all that is missing is the ability to automate what's in a repository and having Ansible apply the configurations to your infrastructure.

A common architectural pattern that we've seen in our user base looks something like this diagram:

DevOps teams do work on their Ansible content and commit it to a Git repo. Those commits are picked up by a CI/CD tool, typically Jenkins, that makes a call out to the Ansible Tower RESTful API to launch an associated playbook or workflow.

This is an effective way of implementing a Git-centric continuous deployment pipeline using Ansible. Once set up, DevOps engineers do all of their work in Git using the toolchain and workflows they're already using. The CI tool picks up the changes and tells Ansible to do its thing.

This is good, but not intuitive and a bit convoluted. We thought we could do better.

Introducing Automation Webhooks

On the Red Hat Ansible team, we saw the benefits of these CI/CD patterns to productivity and overall quality. We also thought we could provide a better user experience for Git-centric deployment pipelines; one that was more straightforward, easier to implement, and simpler to manage. That's why last fall, Automation Webhooks were introduced to Ansible Tower v3.6 as part of the Ansible Automation Platform.

These Automation Webhooks allow you to link a Git repository and Ansible automation natively. Once a repo link is set up, Ansible catches events (Git commits) from the Git system (GitHub, GitHub Enterprise, GitLab). It then uses those events to trigger automation jobs to update projects, manage inventories, and perform deployments, all without requiring yet another tool such as Jenkins.

The benefits of one less tool to use and manage are clear. With these new capabilities, there is no need for an additional CI tool such as Jenkins to monitor repos and launch automation jobs when changes occur.

Setting up your first Ansible Automation Webhook

Let's dive into setting up your first Automation Webhook with Ansible Tower.

First, you provide Ansible Tower with a GitHub or GitLab personal access token (PAT). This credential is used to send status updates back to the Webhook service. (See Credential Types in the Ansible Tower documentation to create one.) While this step is optional, we recommend you take the time to set up this credential.

Next, identify some automation that you want to run when a change happens to a given Git repository. Automation Webhooks are enabled and managed from job or workflow templates in Ansible Tower. In this example, we'll be using a Job Template, so if you don't have one set up for it already, you'll need to do that now.

To enable an Automation Webhook for your job template, scroll down to the "Options" section and tick the "Enable Webhook" box. When you enable a Webhook, other related fields display, prompting for additional information to set up the Webhook source.

Select the Git service to listen for Webhooks. Currently, only GitHub and GitLab are supported by Ansible Tower.

Then, optionally, select the personal access token (PAT) as the credential to use for sending status updates back to the Webhook service. Remember, before you can select it, the credential must already exist, so if you skipped over this step earlier, you need to stop and do it now.

With a Git service and PAT selected, the Ansible Tower UI automatically populates a couple of additional fields for you, including a URL for the Webhook service to POST requests and a generated shared secret key used by the Webhook service to sign payloads sent to Ansible Tower. The "Webhook Key" field has a refresh icon button to its right so you can click to generate a different secret key when needed.

Next, with this unique URL and secret key, go to your Git service and register the Webhook on the repository from which you want Ansible Tower to accept Webhooks. For more information on setting up Webhooks for a supported service, see Working with Webhooks in the Ansible Tower documentation.

With that, you're ready to give your Webhook a try. Both GitHub and GitLab provide a way in their respective UIs to test that your Webhook has been properly configured and is functional. Make a change to your repo's content, push it to the Git repo, and watch Ansible Tower take over in your dashboard.

Automation Webhooks are not just limited to job templates. You can also enable them on your workflow templates. The UI and steps to do that are the same.

We think this is a much more intuitive and straightforward way to enable Git-centric workflows and all their benefits when using Ansible. Anything you can do with Ansible playbooks and Ansible Tower workflows can be run automatically with any commit to a Git repository. Ansible Automation Webhooks provide native support for Git-centric workflows that are intuitive and remove the need for an additional CI/CD tool such as Jenkins to monitor Git repos and launch automation jobs.

But how can we take Ansible Automation Webhooks a step further?

What is GitOps?

Like so many terms that evolve and emerge from the insights and practices of what came before it, finding a definitive meaning to the term "GitOps" is a bit elusive.

GitOps is a workflow whose conceptual roots started with Martin Fowler's comprehensive Continuous Integration overview in 2006 and descends from Site Reliability Engineering (SRE), DevOps culture, and Infrastructure as Code (IaC) patterns.

The term GitOps was coined by Alexis Richardson, CEO and Founder of Weaveworks, so a lot of how I'm going to define GitOps here comes directly from Alexis and Weaveworks. This initial blog post that explains the concept puts some baseline ideas out there but doesn't provide a concise definition. Depending on who you ask, you'll get varying explanations of the term.

After reading and listening to various explanations, I thought this definition was a concise one that captures the essence of the many ways to explain GitOps:

[GitOps] works by using Git as a single source of truth for declarative infrastructure and applications.

-- Weaveworks, "Guide To GitOps"

Often you read that immutable architectures, and even more specifically Kubernetes cluster management, are attributes of GitOps. I suggest that this description is a bit too prescriptive and limiting, so here we treat them as suggestions and preferences, not requirements of GitOps. We'll see why in a bit.

This diagram shows what a typical GitOps workflow looks like from a conceptual level — automating delivery pipelines to roll out changes to your infrastructure when commits are made in a Git repository.

GitOps increases productivity, velocity of deployments, and development. While Git has traditionally been a developer tool, operations staff benefit from the accumulated knowledge and experience of the Git community and the maturity of its ecosystem. There are a plethora of existing tools out there to make using Git more accessible and easier for those new to it.

Using GitOps brings together deployments and operations with development processes and tooling, providing a consistent means of working in your organization.

With the defined state of your infrastructure under Git version control, complete with a useful audit log of all activity, implementing a GitOps workflow improves stability, increases reliability, provides tracking, and allows you to roll back to the previous state or rebuild your systems if you need to recover from a disaster.

GitOps the Ansible way

Previously, we reviewed how you can create Git-centric Infrastructure as Code (IaC) deployment workflows with the Automation Webhooks capabilities in Ansible Tower. GitOps is a more prescriptive workflow of IaC, though, and conceptually looks something like this diagram.

There are a few things to note about using the Red Hat Ansible Automation Platform compared to the typical GitOps pipeline.

For example, Ansible Tower replaces the GitOps "agent" (Operator) that runs on a given cluster and pulls in its configuration (state) from Git. Typically these are Kubernetes Operators like Flux or Eunomia.

Ansible Tower can work with Operators running on a Kubernetes cluster for a push/pull sort of approach. Ansible Tower pushes the configuration to the Operators via a Custom Resource (CR). Then, the Operator pulls in any container images from the registry and handles whatever setup is necessary. The Operators here are made for a specific Kubernetes application or service. They are useful outside of a GitOps pipeline rather than one for all configurations and management of the cluster to facilitate GitOps.

Using Ansible also provides the flexibility to apply GitOps workflow principles to systems other than Kubernetes, such as public/private cloud services and networking infrastructures, because you're not required to use an Operator "agent" on that infrastructure.

Advantages of using Ansible

There are many tools you can use in your GitOps pipelines; however, Ansible provides some unique advantages that make it ideal for these workflows and extending their use beyond Kubernetes and cloud-native systems.

• GitOps beyond Kubernetes. Like Kubernetes, Ansible is a desired state engine that enables declarative modeling of traditional IT systems without scripting through Ansible roles and playbooks. With the k8s module and many others, an Ansible user can manage applications on Kubernetes, on an existing IT infrastructure, or across both with one simple language.
• Agentless GitOps. Consistent with the Ansible way of doing things, there is no requirement for a specialized GitOps Operator (agent) to facilitate the reconciliation of the desired state on your systems.
• Flexibility and freedom. You also have the flexibility and freedom to use the best tools for your needs and better tailor your pipelines to how you want to work. Ansible excels as IT automation glue.
• Existing skills and ecosystem. The same tried and trusted Ansible tooling lets you automate and orchestrate your applications across both new and existing platforms, allowing teams to transition without having to learn new skills.

The benefits of using Ansible in this domain don't end here, however. There are many benefits to using Ansible in a cloud-native Kubernetes environment.

In closing

Using the Red Hat Ansible Automation Platform to implement GitOps pipelines provides unique benefits. Utilizing the Automation Webhook capabilities in Ansible Tower, you can implement agentless GitOps workflows that go beyond just cloud-native systems and manage existing IT infrastructure such as cloud services and networking gear. Using Ansible enables you to tap into the existing Ansible ecosystem with the flexibility and freedom to use the best tools for how you want to work.

We hope you give GitOps with Ansible Automation Webhooks a try and see how beneficial and powerful these tools can be for your organization.

[ Wondering what automation can do for you? Get started with The Automated Enterprise, a free book from Red Hat. ]