Inject external data in policy evaluations with Conftest

In Test Ansible using policy as code with Conftest, I described how to create a basic policy using Open Policy Agent (OPA) to validate an Ansible playbook with the Conftest utility. The policy in that example used a list of software versions hard-coded directly into the policy. However, most policy scenarios involve decoupling the policy from the data that drives the policy decision.

[ Write your first Ansible playbook in this hands-on interactive lab. ]

This article shows you how to separate policy from data. This enables more robust and reusable policy definitions, and this approach can even integrate with external systems to inject data into the policy evaluation process.

Define external data

Policy decisions are rarely made in isolation. Instead, they leverage contextual information about an environment in the decision-making process. This contextual information may change frequently. For example, consider a policy to prevent terminated employees from accessing an internal resource. A policy engine needs two pieces of information to make a decision: the policy itself (terminated employees cannot access internal resources) and contextual information about employees, such as a list of usernames for those who are currently employed by the company.

Attempting to store the contextual data inside the policy itself is messy and difficult to maintain. For example, imagine trying to keep a list of current employees directly in an OPA Rego file. Any change to the current employee list requires knowledge of Rego and modification to the policy code.

[ Read Introduction to policy as code with automation. ]

Conftest supports loading external data to support policy decisions. The -d flag accepts JSON or YAML files to load additional data into the policy engine.

To demonstrate this, I'll extend the example from my previous article to use external data in the decision-making process. First, create a data.yaml file to hold the external data for policy evaluation:

$ touch data.yaml

$ tree
.
├── data.yaml
├── inventory.yaml
├── playbook.yaml
└── policy
    └── main.rego

[ Get the YAML cheat sheet for tips. ]

The data.yaml file contains the list of allowed MariaDB versions defined in a list variable:

$ cat data.yaml
allowed_mariadb_versions:
  - "10.8.6"
  - "10.9.4"

Next, remove the allowed_mariadb_versions list that is hard-coded in the policy:

allowed_mariadb_versions = [
  "10.8.6",
  "10.9.4"
]

Add an import statement to import the data that is passed to Conftest:

import data.allowed_mariadb_versions

The imported data structure uses the same variable name as the previous policy, so no other changes are needed. The final main.rego is shown below:

package main

import future.keywords.contains
import future.keywords.if
import future.keywords.in

import data.allowed_mariadb_versions

deny_unapproved_version contains msg if {
  some task in input.tasks
  package_name := task["ansible.builtin.package"].name
  regex.match("MariaDB-server.*", package_name)

  version := split(package_name, "-")[2]
  not version in allowed_mariadb_versions

  msg := sprintf("Version %v of Mariadb-Server is not allowed", [version])
}

The test functions exactly as it did in the previous article and logs an error if the MariaDB version is not on the allowed list:

$ conftest test playbook.yaml -d data.yaml
FAIL - playbook.yaml - main - Version 10.8.4 of Mariadb-Server is not allowed

1 test, 0 passed, 0 warnings, 1 failure, 0 exceptions

Defining the data in a separate file decouples the policy from the supporting contextual information. Updates, such as the addition of a new MariaDB version, no longer require a direct edit to the code. Instead, an entry can simply be added to the YAML file without any knowledge of Rego. This YAML or JSON file could even be rendered by an external process, such as a database lookup, prior to policy evaluation.

[ Boost your skills with this learning path: Getting started with Red Hat OpenShift Service on AWS (ROSA) ] 

Extend the policy

Decoupling policy from data also allows for more elegant and reusable policy. For example, the current policy can be extended to support other applications. I stated in my previous article that policy should be expressed in plain language first. Modifying the current policy results in the following description:

Applications should be installed only from an approved list of applications and their versions

This policy version is more generic and can support many applications. Coupling this policy definition with external data results in a short but flexible policy definition.

First, define a list of allowed_software in the data.yaml file:

allowed_software:
  - "MariaDB-server-10.8.6-1.fc36"
  - "MariaDB-server-10.9.4-1.fc36"
  - "httpd-2.4.55-1.fc36"

Remove the deny_unapproved_version policy from the previous exercise and define a new deny_unapproved_packages policy. This policy checks to see if the provided package name in the Ansible task is in the list of allowed packages from the data.yaml file. The full main.rego is shown below:

package main

import future.keywords.contains
import future.keywords.if
import future.keywords.in

import data.allowed_software

deny_unapproved_packages contains msg if {
  some task in input.tasks
  package_name := task["ansible.builtin.package"].name

  not package_name in allowed_software
    msg := sprintf("%v is not in the list of approved packages", [package_name])
}

This policy definition is very short, but it is flexible enough to support multiple scenarios without any hard-coded knowledge of the software. Finally, update the Ansible playbook to contain a mix of allowed and forbidden software:

- name: Install software
  hosts: webservers
  tasks:
    - name: Install MariaDB-server
      ansible.builtin.package:
        name: MariaDB-server-10.8.4-1.fc36
        state: present

    - name: Install Apache server
      ansible.builtin.package:
        name: httpd-2.4.55-1.fc36
        state: present

    - name: Install telnet
      ansible.builtin.package:
        name: telnet
        state: present

Running Conftest confirms that the policy was successfully implemented:

• MariaDB-server-10.8.4-1.fc36 generates an error because the version is not in the list of approved packages
• telnet generates an error because it is not in the list of approved packages
• httpd-2.4.55-1.fc36 successfully passes the policy evaluation because it is an allowed package with the proper version

$ conftest test playbook.yaml -d data.yaml
FAIL - playbook.yaml - main - MariaDB-server-10.8.4-1.fc36 is not in the list of approved packages
FAIL - playbook.yaml - main - telnet is not in the list of approved packages

2 tests, 0 passed, 0 warnings, 2 failures, 0 exceptions

Wrapping up

Conftest leverages data provided by YAML or JSON files in its policy evaluation process. Combining well-defined policies with external data sources enables you to define robust and reusable policies. These policies take advantage of contextual information about their environment, and they can integrate with existing data sources to intelligently evaluate policy compliance for your configuration files.

[ Learn about upcoming webinars, in-person events, and more opportunities to increase your knowledge at Red Hat events. ]