As containerized software delivery progresses, Red Hat’s OpenShift has emerged as a beacon of trust in the open source domain. At the heart of this is OpenShift's strategic incorporation of Tekton Chains—a Kubernetes Custom Resource Definition (CRD) controller—to boost supply chain security for nearly every task and pipeline. 

Moreover, OpenShift integrates Cosign extensively, a pivotal tool from the Sigstore family. With Cosign, OpenShift validates that container images are not only built with integrity but are also cryptographically signed, thereby providing an authentication mechanism that asserts the provenance and integrity of the container images. These verifiable signatures offer an additional layer of trust, verifying that no unauthorized changes have been made post-signing.

The synergy of Tekton Chains and Cosign, along with the related tools of Syft and Grype within OpenShift, underscores Red Hat's commitment to driving greater trust and strengthened security in software delivery mechanisms in the modern digital era.

Enhancing image security with Red Hat OpenShift and Cosign

Tekton Chains allows you to manage your supply chain security in Tekton. In its default mode of operation, Chains works by observing all TaskRuns executions in your cluster. Another essential feature is that it can sign OCI registry images with cryptographic key types and services. We will be using Quay as our OCI registry.

Tekton has a few options available to sign your images:

  • x509
  • Cosign
  • KMS
  • EXPERIMENTAL: Keyless signing.

The following example demonstrates how to use Cosign to sign your image. You can find a sample pipeline on Github.

Diagram of a sample pipeline

To start signing things in Chains, you must generate a keypair and instruct Chains to sign with it via an OpenShift secret. If the key is encrypted, chains expect a private key and password to exist in a Kubernetes secret signing-secrets in the openshift-pipelines namespace.

Step 1. Install the Cosign CLI by following these instructions.

Step 2. Login into your OpenShift instance using the oc client on your terminal.

oc login -u <admin> -p <password> <ocp-api-endpoint>

Step 3. Once logged in, run the following command to create a secret called signing-secrets in the openshift-pipelines namespace which Chains will use to sign your images:

COSIGN_PASSWORD=openshift cosign generate-key-pair k8s://openshift-pipelines/signing-secrets

Step 4. You'll also observe a cosign.pub file generated in the present directory. This file will be utilized later to verify the signed image.

Step 5. Configure Tekton Chains by running the following commands:

cat <<EOF >> chains.yaml
apiVersion: operator.tekton.dev/v1alpha1
kind: TektonChain
metadata:
  name: chain
spec:
  artifacts.oci.storage: oci
  artifacts.taskrun.format: in-toto
  artifacts.taskrun.storage: oci
  config: {}
  targetNamespace: openshift-pipelines
  transparency.enabled: true
EOF
oc apply -f chains.yaml -n <pipelines-namespace>

Step 6. Run the following command a few times to check if the deployment is available:

oc get deployment tekton-chains-controller -n <pipelines-namespace>

Step 7. Wait for the AVAILABLE column to display 1

NAME                              READY   UP-TO-DATE   AVAILABLE   AGE
tekton-chains-controller   1/1          1                       1                    106m

Step 8. For Tekton Chains to sign an image, these TaskRun results are required:

  • *IMAGE_URL – The URL to the built OCI image
  • *IMAGE_DIGEST – The Digest of the built OCI image

The asterisk (*) indicates any expression. For example, if both XXX_IMAGE_URL AND XXX_IMAGE_DIGEST are correctly formatted to point to an OCI image, then chains will pick up on it and try to sign the image.

To get more detail about the process, read this documentation.

The buildah ClusterTask is an example of a pipeline task used to build a container image. It also conveniently sets these results in its TaskRun, which Tekton Chains can then use to sign the image. To create and sign your new image, use this task.

Step 9. You can use Red Hat Advanced Cluster Security for Kubernetes to confirm the integrity of the container images in your clusters by verifying image signatures against preconfigured keys.

You can also create policies to block unsigned images and images that do not have a verified signature and then enforce the policy using the Red Hat Advanced Cluster Security admission controller to stop unauthorized deployment creation.

To verify your image signature in Red Hat Advanced Cluster Security, follow these instructions.

Use the contents of the cosign.pub file created above for your signature integration.

You're also required to provide Red Hat Advanced Cluster Security access to your container registry to scan your image. More details can be found here.

Step 10. Once the policy is set up, create a pipeline task to invoke the roxctl command on your new container image:

./roxctl --insecure-skip-tls-verify image check --endpoint <acs-route> --image <container-image-repo>:<tag>

Step 11. Your pipeline now clones your source code, builds your container image, and runs an ACS check on your image. The pipeline is: clone-source -> build-sign-image -> acs-image-check

Step 12. Run your pipeline.

Step 13. Tekton Chains creates annotations on your TaskRun when signing your images. This confirms that your image was signed during this step in your pipeline. The annotations to look out for are chains.tekton.dev/signed and chains.tekton.dev/transparency your signed image:

cosign verify --key cosign.pub <container-image-repo>:<tag>

You can also log into Quay to confirm your image was indeed signed. Your image tag should have a black shield indicating your image was signed using Cosign:

Screenshot showing a list of repository tags

Wrapping up the security conversation

The deeper you delve into the nuances of container security, the clearer it becomes that Red Hat is one of the leaders in fortifying containerized software delivery. This installment follows our initial discussion on the broad approach to containerization security. In that post, we highlighted the pivotal role of Red Hat Advanced Cluster Security (ACS) and Cosign policy. The policy confirms that ACS checks and guarantees that each image has been signed, reinforcing trust at every layer. 

Stay tuned as we continue to unpack the latest advancements and strategies in open source security. Your journey with Red Hat is a testament to our shared commitment to building a safer, more secure digital landscape.


About the authors

As a seasoned technology professional and Team Lead for the APAC content development team, my role is dedicated to optimizing sales and marketing efforts for Red Hat products. I excel in crafting detailed, comprehensive demos that illuminate the unique selling propositions of our product range, addressing both technical and non-technical audiences. In addition, I work hand-in-hand with our sales and marketing teams to produce engaging marketing materials that align with current market trends and customer needs, ensuring our messaging strikes a chord with our target demographic. With a strong foundation in Red Hat technologies and a knack for translating complex technical features into relatable benefits, my work plays a pivotal role in supporting the sales process, boosting brand affinity, and ultimately driving the market presence of Red Hat products.

Read full bio

I started at Red Hat in 2021 as a Senior Architect and currently responsible for delivering sales demos using the Red Hat product set on our demo platform demo.redhat.com

Read full bio