Terraforming  Azure Red Hat OpenShift (ARO) with GitHub Actions

Introduction

By combining Infrastructure as Code with Continuous Integration and Continuous Delivery and/or Deployment  CI/CD in cloud platforms, organizations can achieve a highly automated and streamlined software development and delivery process. Developers can use code to define and provision infrastructure resources, and use CI/CD to continuously integrate code changes and test the software. This approach can help organizations to achieve faster and more reliable software delivery, as well as improve the overall quality of their software.

Many organizations use GitHub Actions and Terraform to deploy solutions in Azure as best practices to achieve collaboration, automation, scalability, and security.

What will be covered

In this blog post, we are going to learn how to deploy Azure Red Hat OpenShift (ARO) with Terraform and integrate it with Terraform Cloud and GitHub Actions. If you want to learn about Azure Red Hat OpenShift, you can get more information here.

Note: Azure-Samples/aro-azapi-terraform has been used for Terraform code as referenced in this post. 

Dataflow

1. Create a new branch from the main and modify the code. As an example changing the work node instance type or subnet address.
2. Create a pull request (PR) to the new branch.
3. GitHub Actions workflow will be triggered to guarantee that your code is correctly formatted, consistent internally, and results in a secure infrastructure.
4. Create PR to merge into the main branch.
5. GitHub Actions workflow will be triggered to run a terraform plan. The result of Terraform plan will be accessible in both GitHub Actions and Terraform cloud workspace. 
6. After reviewing PR by the team and merging to the main branch, Terraform apply will execute.
7. Terraform will deploy Red Hat OpenShift to Azure. 

Workflow

For maintaining code quality, preventing accidental changes, enforcing policies, collaborating more effectively and as a best practice, it’s recommended that developers not push directly to the main branch.  

By using protected branches, teams can ensure that critical branches are properly managed and maintained, which can lead to a more stable and reliable codebase over time.

The infrastructure team should create a new branch from main and apply changes to it, then push to feature/development branch. Pushing to branch will trigger Terraform-UnitTests.

After running Terraform-UnitTests successfully, the infrastructure team can create a pull request which will trigger Terraform-Push. 

We will use two workflows:

• Terraform-UnitTests.yml: The purpose of this workflow is to run unit tests on push into any branch.  As part of this workflow’s Terraform validation, the format and security scans will be checked. 
• Terraform-Push.yml: This workflow has two phases, one for push and one for merge. 
  Based on each push from feature/development branches to the main branch, this workflow will trigger and run the terraform plan command. After a successful push, when a merge request is submitted this workflow will trigger the terraform apply command.

Prerequisites

• Azure Account with active subscription
• Terraform Cloud account
• GitHub account
• Visual Studio Code + Git + Azure CLI + Terraform CLI installed in local workstation
• Red Hat account  

Steps

1. Install Azure CLI

Azure CLI is a command-line interface for managing resources in Microsoft Azure, which is Microsoft's cloud computing platform. 

Install Azure CLI in your local workstation. 

2. Increase limits by VM series

Increasing limits by VM series for Azure Red Hat OpenShift installation is necessary to ensure that your Azure Red Hat OpenShift cluster has the resources it needs to operate efficiently and reliably. 

Standard DSv3 Family vCPUs = 150  
Total Regional vCPUs = 200

3. Get a Red Hat pull secret

Navigate to your Red Hat OpenShift cluster manager portal and sign-in. Download the pull secret. We will use this secret in the Terraform Cloud while provisioning the Azure Red Hat OpenShift cluster. 

4. Fork and clone the GitHub repository

Fork the following repository in your GitHub account  and clone it in your local workstation.

https://github.com/DexHat/aro-terra-gitactions

After cloning in the local workstation update your .gitignore with following information:

#**Secrets**
pull-secret*
variables_secrets*

5. Set variables and runthe create.sh script

Set following variables for the create.sh script:

#resourcePrefix="<azure-resources-name-prefix>"
#aroDomain="${resourcePrefix,,}"
#aroClusterServicePrincipalDisplayName="${resourcePrefix}-aro-sp-${RANDOM}"
#pullSecret="${RedHat-PullSecret-Path}"
#location="${ARO-location}"

Run the az login command to connect to Azure:

az login --use-device-code

Run  the create.sh file with following command:

sh create.sh

It will:

• Create variables-secrets file with all variables for Azure Red Hat OpenShift cluster.
• Register necessary service providers.
• Create service principal and assign contribute and system administrator role to it.
• Generate variables for Terraform Cloud, GitHub Actions, and tfvars.

6. Terraform Cloud sign-up

To integrate between Terraform and GitHub Actions, we are going to use Terraform Cloud. 

First, sign up for a new Terraform Cloud account. Next, choose an organization or create an organization.

Create a new workspace and select API-driven as a workflow type.

7. Copy and set variables and secrets

Copy the following variables and secrets from the variables_secrets file and set them in Terraform Cloud workspace level.

Terraform variables:

• aro_cluster_aad_sp_client_id = Service Principal Application (client) ID

• aro_cluster_aad_sp_client_secret =  Service Principal Secret Value

• aro_cluster_aad_sp_object_id = Service Principal Object ID

• aro_rp_aad_sp_object_id = Azure Red Hat OpenShift Resource Provider Object ID

• pull_secret = xxxxx

Environment Variables

• ARM_CLIENT_ID =  Service Principal Application (client) ID

• ARM_CLIENT_SECRET = Service Principal Secret Value

• ARM_SUBSCRIPTION_ID = Azure Subscription ID

• ARM_TENANT_ID = Azure Tenant ID 

For more information about sensitive variables in Terraform, please check Protect Sensitive Input Variables.

Since we are going to use Terraform Cloud, we will set all variables at the workspace level.

8. Install Terraform CLI

Terraform CLI is a powerful tool for managing infrastructure as code, allowing users to version control infrastructure changes, collaborate more effectively, and automate the deployment and management of infrastructure resources. Install Terraform CLI from here.

Integrating Terraform Cloud and GitHub Actions

After installing Terraform CLI locally, run terraform login to create a token.

The generated token will be used to integrate with GitHub Actions as CI/CD pipeline.

Under the GitHub repository - Secrets and Variables - Actions, create the TF_API_TOKEN secret and copy the value from the previous step to here. 

In GitHub, under Environments, create a Development environment.

You can deploy Azure Red Hat OpenShift cluster with different sizing and variables under different environments like production, staging, and development.

Copy the following variables and secrets from the variables_secrets file and set them under the GitHub repository.

• ARM_CLIENT_ID =  Service Principal Application (client) ID
• ARM_CLIENT_SECRET = Service Principal Secret Value
• ARM_SUBSCRIPTION_ID = Azure Subscription ID
• ARM_TENANT_ID = Azure Tenant ID 

Copy the following variables and secrets from the variables_secrets file and set them in the Development/tfvars file on your local workstation.

• domain
• location
• resource_group_name
• resource_prefix
• virtual_network_address_space
• master_subnet_address_space
• worker_subnet_address_space

9. Push and merge to the GitHub repository

After creating/updating Development/tfvars,  push it to your GitHub repository. This will trigger Terraform-UnitTests.yml.

Terraform-Push.yml will be triggered when you merge to the main branch. 

Cluster Information

You can get cluster information using this approach:

Conclusion

In conclusion, the use of GitHub Actions and Terraform Cloud together can provide a powerful solution for deploying and managing Azure Red Hat OpenShift (ARO) clusters. GitHub Actions provides a way to automate and streamline the process of managing code changes and deployments, while Terraform Cloud provides a way to manage infrastructure as code in a scalable and efficient manner. Azure, as a cloud provider, offers a robust platform for hosting and managing Azure Red Hat OpenShift clusters. By leveraging these tools and services, organizations can deploy and manage Azure Red Hat OpenShift clusters more efficiently, with greater consistency and control, and with the ability to easily scale up or down as needed.