Introducing OpenShift Pipelines

OpenShift Pipelines is Generally Available (GA) as of May 3rd, 2021, and it provides a cloud-native continuous integration solution based on Tekton, a Continuous Delivery Foundation (CDF) project.

OpenShift Pipelines provides a kubernetes-native CI/CD framework to design and run your pipelines, and is designed to run each step of the CI/CD pipeline in its own container, allowing each step to scale independently to meet the demands of the pipeline.

Before diving into the OpenShift Pipelines features, let’s first have a quick reminder of the main Tekton concepts.

Kubernetes-native CI/CD concepts

By extending Kubernetes/OpenShift with Custom Resource Definitions (CRDs), OpenShift Pipelines makes CI/CD concepts such as a “pipeline”, a “task”, a “step” natively instantiatable so it can use the scalability, security, ease of deployment capabilities of Kubernetes.

So first, let’s learn more about these concepts brought by Tekton.

                                                   Figure 1.  Tekton Concepts

Basically, the main concepts are as follows, split into concepts that define the pipeline and others that run the pipeline.

Concepts that define the pipeline

• Pipeline: the definition of the pipeline and the Tasks that it should perform
• Task: a reusable, loosely coupled number of steps that perform a specific task (e.g. building a container image)

Concepts that run the pipeline

• PipelineRun: the execution and result of running an instance of a pipeline, which includes a number of TaskRuns
• TaskRun: the execution and result of running an instance of a Task

For more information on these concepts you can check the official documentation.

So now that we understand the fundamentals of Tekton, what is OpenShift Pipelines and what’s in it for us?

OpenShift Pipelines, what’s in it ?

OpenShift Container Platform is the industry’s leading Enterprise Kubernetes platform, and it brings out-of-box many features for developers, among which CI/CD capabilities.

OpenShift Pipelines is based on the Tekton project, and offers a native integration with the OpenShift platform to provide a smooth experience for the developers.

OpenShift Pipelines installation with the Operator

Since it’s supported by an Operator, OpenShift Pipelines is very easy to install and upgrade, making its administration a treat so you can focus on building value on top of it.

It is available in OpenShift Container Platform’s OperatorHub, the internal Marketplace of over 450 operators, as follows:

The installation is very straightforward, and it is installed as a cluster-wide operator, making it available to all projects.

It will also extend OpenShift’s underlying Kubernetes with some new CRDs that allow to bring even more versatility in terms of configuration, integration with the OpenShift console, etc.

When a new release is made available in the OperatorHub, you can (as an administrator) choose to upgrade to the next version by selecting the appropriate “upgrade channel”.

A rich UI and fully integrated experience within the OpenShift Console

Although Tekton extends a traditional Kubernetes with these new CI/CD concepts, it is still a heavy lift to be able to smoothly create and run pipelines without writing thousands of lines of YAML, that’s why we decided to provide a very rich UI within OpenShift Console to both run and visualize pipelines as they are being executed (or past runs), but also to graphically design your pipelines while OpenShift creates the corresponding YAML files in the background for you.

In the following example, we can see a graphical representation of a pipeline that has been executed on OpenShift, and we can have access to the logs and events associated with all the tasks

Figure 2. Pipelines in the OpenShift Console

If we wanted to know what happened within a specific task, we can access its logs as follows:

To make the developer’s life even easier, OpenShift Pipelines allows you to graphically build your pipelines directly from the console, taking away the need to be a YAML black-belt guru to create your first Tekton-based pipeline

Figure 3. Graphically build your pipelines from the OpenShift Console

Of course if you’re a YAML black-belt and still want to edit advanced properties, you can still do that directly from the console, as seen below:

Figure 4. YAML Samples and snippets are also available for YAML lovers 

The great thing about the OpenShift experience is that even if you wanted to go the YAML way, you can still rely on some accelerators such as the built-in snippets or samples, making it more productive to create your YAML pipelines. And you can also integrate your own enterprise blueprints of samples and snippets, making them available for your developers. In fact, we added a Custom Resource Definition called ConsoleYAMLSamples that allows you to do just that.

Triggering Pipelines from events

When creating your pipelines, you’ll probably want to link their execution to some external events, or what we call a Trigger in Tekton terms, such as a Github Push or Pull Request, or Gitlab, and OpenShift Pipelines happens to offer that to you out-of-the-box, by supporting many vendor solutions, such as Github, Gitlab, BitBucket, etc:

Figure 5. You can easily add triggers for the OpenShift Console 

By using the UI to create these triggers, OpenShift will create several other needed concepts such as EventListeners, TriggerTemplates, but we will get to these concepts later in a soon to come blog post, just stay tuned. Meanwhile, you can read more about these in the official documentation.

Out-of-the box reusable Tasks, and customization

OpenShift Pipelines provides out-of-the-box dozens of Tasks to be used in your pipelines, ranging from things like cloning code from a repository, to building your application for several languages such as java, dotnet core, python go, nodejs, or using build tools like maven, or deploying your application etc. The list of tasks can be seen in the OpenShift Console, under the ClusterTasks section in the Pipelines -> Tasks menu

Figure 6. Dozens of out-of-the-box Tasks are available with OpenShift Pipelines

Extending the list of available tasks is very easy, as all it requires is adding “ClusterTasks” to the cluster, and you can find hundreds of Tasks on TektonHub, the public repository to share Tekton tasks.

Figure 7. TektonHub, the public repository to find reusable Tekton Tasks and Pipelines

Integration with the developer’s IDE

Developers using command line and IDEs can take advantage of Tekton CLI, the Tekton

extension for Visual Studio Code and Tekton plug-in for IntelliJ, to interact with pipelines without leaving their environment and create, start, view, and perform actions on the cluster directly from the command line.

Figure 8. VSCode’s extension for OpenShift Pipelines

Finally, to get a sneak peek of all these features, you can check the following video:

For further information, you can check out the following resources:

• OpenShift Pipelines and OpenShift GitOps are GA blog post
• OpenShift Pipelines product page: https://www.openshift.com/learn/topics/ci-cd
• OpenShift Pipelines product documentation: https://docs.openshift.com/container-platform/4.7/cicd/pipelines/op-release-notes.html
• OpenShift Pipelines tutorial on https://learn.openshift.com/gitops
•  A great demo by Siamak Sadeghianfar, Product Manager of OpenShift Pipelines: https://github.com/siamaksade/tekton-cd-demo

I hope this post gave you more information on some of the OpenShift pipelines features, and stay tuned for a “getting started with OpenShift Pipelines and OpenShift GitOps” blog post that will take you through a practical usage of these two solutions together. Thank you for reading!