Argo CD and Release Management with Helm Charts and ApplicationSets

The Problem

With Argo CD or OpenShift GitOps, deploying Helm Charts and leveraging the power of templating is simple. However, while working with different cluster environments, the question sometimes arises: How can I make sure that a new version of a Chart is first deployed on the DEV-environment only before it is installed on the PROD-environment?

In other words, if a new Helm Chart version has been released, it should be tested before it has any unwanted side effects on the production environment.

For example, imagine you have a Helm Chart named my-super-chart with the version 1.0.0. It is installed on the dev and production clusters. A new version with some changes is created, increasing the version number to 1.0.1. You must make sure that this change is first deployed on the dev cluster only so it can be verified. If everything is fine, it might be rolled out to production.

The simple solution is to use a different targetRevision for each cluster in the Application objects of Argo CD. But how can you accomplish this when there is an ApplicationSet in place? How can you configure the targetRevision inside the ApplicationSet? Make sure you define the targetRevision for each cluster separately.

Prerequisites

The following requisites are mandatory to work with releases:

• Helm Chart Repository. This might be any artifactory or even GitHub. If you want to know how I use GitHub to release my charts, check out the GitHub actions in my repository.
• Multiple clusters, such as a development cluster (dev) and a production cluster (prod).

ApplicationSet

The following ApplicationSet uses the list generator to roll out a configuration (etcd encryption) to multiple clusters. It will create an Application object per cluster in Argo CD. The clusters are dev and prod. In addition, it defines the setting chart_version. This triggers which version will be deployed on which cluster. Dev will use the latest version 1.0.16, while the production environment still uses 1.0.13. Once the tests are successful, you can change the version for production to the required number.

The magic happens in the last line of the ApplicationSet targetRevision, which reads '{{ chart_version }}'. This entry will create the Application objects pointing to the version defined in the list generator.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: etcd-encryption
  namespace: openshift-gitops
spec:
  generators:
    - list:
        elements:
          - chart_version: 1.0.16
            cluster: dev
            url: 'https://kubernetes.default.svc'
          - chart_version: 1.0.13
            cluster: prod
            url: 'https://api.ocp.aws.ispworld.at:6443'
  template:
    metadata:
      name: '{{ cluster }}-etcd-encryption'
    spec:
      destination:
        namespace: default
        server: '{{ url }}'
      info:
        - name: Description
          value: Enable Cluster ETCD Encryption
      project: '{{ cluster }}'
      source:
        - chart: generic-cluster-config
          repoURL: 'https://charts.stderr.at/'
          targetRevision: '{{ chart_version }}'

Applications

The snippets of the two Application objects that the ApplicationSet creates look like this:

DEV environment
[...]
spec:
  destination:
    namespace: default
    server: 'https://kubernetes.default.svc'
  info:
    - name: Description
      value: Enable Cluster ETCD Encryption
  project: dev
  source:
    - chart: generic-cluster-config
      repoURL: 'https://charts.stderr.at/'
      targetRevision: 1.0.16

PROD environment

[...]
spec:
  destination:
    namespace: default
    server: 'https://api.ocp.aws.ispworld.at:6443'
  info:
    - name: Description
      value: Enable Cluster ETCD Encryption
  project: prod
  source:
    - chart: generic-cluster-config
      repoURL: 'https://charts.stderr.at/'
      targetRevision: 1.0.13

Wrap up

With this simple change, it is possible to define the Chart version in the ApplicationSet per cluster and make sure that only the desired version is deployed onto the appropriate cluster.