Configuring Red Hat Ansible Automation Platform with GitOps

Red Hat Ansible Automation Platform comes in handy when organizations start to implement Infrastructure as Code and GitOps concepts. But what about the automation of the platform itself? 

Before we get started it’s good to clarify some product component naming. Ansible Tower is being renamed to Automation Controller in the upcoming Ansible Automation Platform 2 release. While things are being changed, you can see both words “tower” and “controller” used interchangeably in various contexts including roles and collections. For more information on Ansible Automation Platform please refer to this Knowledge Base Article.

Almost all of the assets in Ansible Tower can be backed with SCM-based repositories. Yet, there's a little gap in how this content (Job Templates, Projects, Inventories, etc.) is initially created and subsequently managed. This becomes especially visible on large deployments. For example, a Project in Ansible Tower can be backed by a Git repository with the automation code. 

But, when making a Project, we configure other settings such as "Update Revision on Launch", which are stored only in Ansible Tower. The same applies to Inventories and Job Templates: they can be linked to Projects, but there isn’t any dedicated place in Git to store their own settings. 

In other words, there are two complementary sources of truth: the repository where, say, a playbook is stored and the settings in Ansible Tower WebUI that make this playbook become an actual Job Template.

This article will try to fill this gap using automation through Ansible, specifically the controller_configuration collection from the Red Hat Automation Community of Practice. 

It's based on either upstream awx.awx collection or its supported variant ansible.tower (for Red Hat Ansible Automation Platform v2+ use the new ansible.controller collection). To learn more about it, see Configuring Ansible Tower with the Tower Configuration Collection blog post. More information on Red Hat Communities of Practice can be found in this blogpost and on GitHub.

This collection allows you to manage pretty much everything about Ansible Tower: configuration, users, inventories, templates, etc. Entities are described as lists of properties that match the awx.awx’s module parameters, e.g., the Projects would look like:

controller_projects:
  - name: Test Project
    scm_type: git
    scm_url: https://git.example.org/ansible/example1.git
    scm_branch: main
    scm_clean: true
    description: Test Project 1
    organization: Satellite
    wait: true
    update: true
  - name: Test Project 2
    scm_type: git
    scm_url: https://git.example.org/ansible/example2.git
    description: Test Project 2
    organization: Satellite
  - name: Test Inventory source project
    scm_type: git
    scm_url: https://git.example.org/ansible/example3.git
    description: ansible-examples
    organization: Satellite

Awesome! Now we can reflect everything that we see in the WebUI as code, having just one source of truth. The next question is: “How to organize this code?”

Technically, you can describe the entire platform in one repository. The good thing about it is that you can provision all assets into the platform with one playbook run. It sounds close to having a DR plan, doesn’t it? The collection even has a playbook you can use.

At the same time, each SCM-backed asset, like an Inventory and its Source Project, will probably have its own maintainers and could be scoped to an Organization. One big repository can make code maintenance and access management challenging. 

Let’s look again at the code snipped above, specifically at the “scm_url” parameter pointing at some repository. What if we implant the definition of each asset directly into its SCM repository?

For an SCM-backed Inventory, one would need at least the following:

• inventory

• inventory source pointing to a project

• inventory source project pointing to an SCM repo

• credential to authenticate against the SCM repo

What we can do is describe all of these entities in a simple file called controller_configs.yml right next to the usual host_vars, group_vars and inventory.yml:

├── group_vars
│   ├── all
│   │   └── main.yml
│   └── db
│       └── main.yml
│
├── host_vars
│   └── db1.example.org.yml
│
├── inventory.yml
└── controller_configs.yml

Then your controller_configs.yml may look as follows:

controller_credentials:
  - name: git_auth
    credential_type: Source Control
    organization: Default
    inputs:
      username: ${GIT_USERNAME}
      password: ${GIT_PASSWORD}

controller_projects:
  - name: example_inventory_source_project
    organization: Default
    scm_url: ${SCM_URL}
    scm_branch: ${SCM_BRANCH}
    scm_type: git
    scm_clean: false
    scm_delete_on_update: false
    scm_update_on_launch: true
    scm_credential: git_auth
    allow_override: false

controller_inventory_sources:
  - name: example_inventory_source
    source: scm
    source_path: inventory.ini
    source_project: example_inventory_source_project
    inventory: example_inventory
    overwrite: true
    overwrite_vars: true
    update_on_launch: false
    update_on_project_update: true

controller_inventories:
  - name: example_inventory
    organization: Default
    instance_groups: []

Another, perhaps a cleaner way, would be to use a subfolder and split the objects into smaller files:

└── controller_configs.d
    ├── credentials.yml
    ├── inventories.yml
    ├── inventory_source_projects.yml
    └── projects.yml

What did we achieve? Now everything about this inventory is described in this one repository. Whoever is responsible for its maintenance is in full control of both: the inventory and how it’s configured inside Ansible Tower.

It’s time to make this configuration propagate to Ansible Tower. This is where we start calling it GitOps. We’ll do this with the example of GitLab CI/CD, but it should be easy enough to adapt it to other tools. 

First of all, we need a playbook. You can write your own playbook or use one from the collection. With ansible 2.11, you may even call it directly from the collections namespace. Feed it with some controller_configs, and it will populate them into Ansible Tower.

After that, we need to make a container image that can run the “ansible-playbook” command and has the necessary collections. An example of such an image built from upstream bits can be found here. The image itself is hosted at quay.io/anestero/cicd-ansible. Feel free to use this image for tests. Make sure to use supported base images, install ansible and collections from supported channels when building images for production use.

The next step is to make our CI/CD tool run automatically every time someone pushes code to our Inventory repository. In the case of GitLab, the .gitlab-ci.yml is responsible for this:

controller_configuration:
  image: quay.io/anestero/cicd-ansible:ansible29-latest
  # For newer versions of AWX/AAP use the latest (ansible 2.11) image
  # image: quay.io/anestero/cicd-ansible:latest
  variables:
    ANSIBLE_FORCE_COLOR: 'true'
    ANSIBLE_HOST_KEY_CHECKING: 'false'
  before_script:
    - mkdir ~/.ssh
    - chmod 700 ~/.ssh
    # Inject values from GitLab CI/CD vars
    - mkdir -p configs/controller_configs.d
    - for f in controller_configs.yml controller_configs.d/*; do [[ -f ${f} ]] && cat ${f} | envsubst > configs/${f}; done
    - export CONTROLLER_CONFIGS_DIR=${PWD}/configs
  script:
    - ansible-playbook ~/.ansible/collections/ansible_collections/redhat_cop/controller_configuration/playbooks/configure_controller.yml
    # For newer versions of AWX/AAP that run ansible 2.11 simply call the playbook from its namespace
    # - ansible-playbook redhat_cop.controller_configuration.configure_controller

We’ve got everything we need. Let’s have a look at one example of how the entire configuration can be organized in the GitLab repositories:

There are two scopes for the configuration:

1. Global scope defines settings and assets for the entire automation controller. Organizations are also defined there.

2. Organization scope defines settings and assets limited to one organization. The entire folder structure for the organization can be automatically created as a part of the global settings delivery pipeline. Each organization subgroup can have its own “Owners”, “Maintainers”, “Developers”, etc.

Takeaways & Next Steps

As shown above, with the help of the controller_configuration collection we can fully automate Ansible Automation Platform. We created one source of truth that defines the configuration of the entire Automation Controller completely in git. 

Having GitLab, we’re also leveraging its CI/CI to deliver this code automatically. Of course, this could be adapted to any other git distributions and pipeline implementations. The main thing is that we use one tool - git - and one language - Ansible - to automate the Automation Platform.

Examples and other assets are available on my GitHub repository for Ansible Tower GitOps.

If you want to learn more about Red Hat Ansible Automation Platform and how Red Hat Services can help accelerate your network automation initiatives, you can check out these resources:

• E-book: The automated enterprise: Unify people and processes

• Get a trial of Red Hat Ansible Automation Platform

• Lead with automation and connect your teams with Red Hat Services