Webhooks can be triggered whenever specific events on GitHub send change information to OpenShift Pipelines for integrating a CI/CD workflow. In the process, you should set up your server to receive and manage the payload from the Internet by adding triggers to a pipeline. If your pipeline is in a restricted network and does not allow access from the Internet (GitHub), consider poll-based change detection for integrating GitHub on the Internet instead.
Poll-based change detection is a kind of polling trigger, which is an event that periodically makes a call to your Git repository to look for new changes. This article will take you through configuring polling triggers in OpenShift Pipelines (Tekton) in a restricted network.
Workflow of pull-based triggers
Basically, a polling trigger is a pull-based workflow, while webhooks are push-based. In other words, polling triggers periodically initiate an event over an interval to determine if there is any change, whereas webhooks respond to a push of changes from the repository.
If you have a private GitLab repository in your disconnected network, you can configure the polling triggers for your pipeline CI/CD using Repository Mirroring (which mirrors all changes over an interval with another repository). A push event usually triggers webhooks. The following diagram shows the workflow based on GitLab:
However, GitLab is overkill for some use cases, so you should implement polling triggers using CronJob with Bash scripts. The diagram above illustrates this in Option 2.
Polling triggers using CronJob
I prepared a sample application for testing a pipeline run through Creating CI/CD solutions for applications using OpenShift Pipelines as follows. In this section, I skipped a secret token configuration in the Trigger YAML for simpler tests. In a restricted network, security would not be a problem.
// Create a project for the sample application
$ oc new-project pipelines-tutorial
// Creating pipeline tasks
$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.9/01_pipeline/01_apply_manifest_task.yaml
$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.9/01_pipeline/02_update_deployment_task.yaml
// Assembling a pipeline
$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.9/01_pipeline/04_pipeline.yaml
// Adding triggers to a pipeline
$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.9/03_triggers/01_binding.yaml
$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.9/03_triggers/02_template.yaml
$ oc create -f - <<EOF
apiVersion: triggers.tekton.dev/v1beta1
kind: Trigger
metadata:
name: vote-trigger
spec:
serviceAccountName: pipeline
interceptors:
- ref:
name: "github"
params:
- name: "eventTypes"
value: ["push"]
bindings:
- ref: vote-app
template:
ref: vote-app
EOF
$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.9/03_triggers/04_event_listener.yaml
// Running a pipeline
$ tkn pipeline start build-and-deploy \
-w name=shared-workspace,volumeClaimTemplateFile=https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.9/01_pipeline/03_persistent_volume_claim.yaml \
-p deployment-name=pipelines-vote-ui \
-p git-url=https://github.com/openshift/pipelines-vote-ui.git \
-p IMAGE='image-registry.openshift-image-registry.svc:5000/$(context.pipelineRun.namespace)/pipelines-vote-ui' \
--use-param-defaults
$ oc get pod -n pipelines-tutorial
NAME READY STATUS RESTARTS AGE
build-and-deploy-run-tbks8-apply-manifests-pod 0/1 Completed 0 3m38s
build-and-deploy-run-tbks8-build-image-pod 0/1 Completed 0 4m11s
build-and-deploy-run-tbks8-fetch-repository-pod 0/1 Completed 0 4m23s
build-and-deploy-run-tbks8-update-deployment-pod 0/1 Completed 0 3m24s
el-vote-app-68fcd9675f-hdw6j 1/1 Running 0 10m
pipelines-vote-ui-676bccc85-m4rsp 1/1 Running 0 3m20s
After confirming that the sample application pipelines-vote-ui is working well, replace the above GitHub repository with the one you have forked from the front-end pipelines-vote-ui for your tests (in my case, https://github.com/bysnupy/pipelines-vote-ui). I will apply a change to my repository to check whether the pipeline kicks off or not in the following demonstration.
Let’s look at the details of the following resources to implement simple polling triggers with scripts. Simply comparing the previous and current revision sha256 hash values using the git command is the main logic to determine triggering the pipeline, and the latest sha256 hash would be stored in a PV volume for the next checks. The CronJob requests a push event using the curl command to an EventListener on behalf of the GitHub repository webhook for triggering a pipeline run.
This approach allows us to reuse existing TriggerBinding and TriggerTemplate resources. Regardless, you can consider using tkn CLI instead of the curl command in CronJob for your needs. The tkn CLI can customize more than the curl command in your pipeline flow.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: repolist-pvc
namespace: pipelines-tutorial
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
---
apiVersion: batch/v1
kind: CronJob
metadata:
name: polling-triggers
namespace: pipelines-tutorial
spec:
schedule: "*/1 * * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: polling-triggers
volumeMounts:
- name: repolist
mountPath: /repolist
image: registry.redhat.io/rhel8/toolbox:latest
env:
- name: REPONAME
value: pipelines-vote-ui
- name: REPOBASEURL
value: https://github.com/bysnupy
- name: REPOBRANCH
value: master
- name: REPOURL
value: https://github.com/bysnupy/pipelines-vote-ui
- name: BASEDIR
value: /repolist
- name: EVENTLISTENERSVC
value: "http://el-vote-app.pipelines-tutorial.svc.cluster.local:8080"
- name: JSONTEMPLATE
value: '{"object_kind": "push","event_name": "push","head_commit": {"id": "GITHUBREPOREV"},"repository": {"name": "GITHUBREPONAME","url": "GITHUBREPOURL"}}'
command:
- /bin/sh
args:
- -c
- |
set -eu
# revision initialization
_current_revision=$(git ls-remote --heads ${REPOBASEURL}/${REPONAME} ${REPOBRANCH} | awk '{print $1}')
_prev_revision=${_current_revision}
# if there is not existing previous revision data file, creating new revision data file with current revision
test -f ${BASEDIR}/${REPONAME}.sha256 && _prev_revision=$(cat ${BASEDIR}/${REPONAME}.sha256) || echo ${_current_revision} > ${BASEDIR}/${REPONAME}.sha256
# generating JSON data
_jsondata=$(echo ${JSONTEMPLATE} | sed -e "s=GITHUBREPOREV=${_current_revision}=" -e "s=GITHUBREPONAME=${REPONAME}=" -e "s=GITHUBREPOURL=${REPOURL}=")
# check if there are any changes through comparing previous and current revisions.
# If there are any changes, trigger a new pipeline using curl and json data.
test "${_current_revision}" != "${_prev_revision}" && echo ${_current_revision} > ${BASEDIR}/${REPONAME}.sha256 &&
curl -s -X POST -H 'Content-Type: application/json' -H 'X-GitHub-Event: push' \
-d "${_jsondata}" ${EVENTLISTENERSVC} ||
echo "No changes"
restartPolicy: Never
volumes:
- name: repolist
persistentVolumeClaim:
claimName: repolist-pvc
For more details of GitHub webhook HTTP POST payloads for modification of curl options, please refer to Webhook events and payloads.
Triggering a pipeline run using polling triggers
When a change event occurs in the Git repository, the aforementioned polling triggers CronJob checks it periodically. After becoming aware of new data, send an event payload (JSON data) to the privately exposed EventListener service using the curl command. The EventListener service of the application processes the payload and passes it to the relevant TriggerBinding and TriggerTemplate resource pairs. The TriggerBinding resource extracts the parameters, and the TriggerTemplate resource uses these parameters and specifies how to create the resources. This process may rebuild and redeploy the application.
In this section, you push an empty commit to the front-end pipelines-vote-ui repository, triggering the pipeline run.
// Check the polling triggers pod logs before changes
$ oc get pod -n pipelines-tutorial
NAME READY STATUS RESTARTS AGE
el-vote-app-68fcd9675f-hdw6j 1/1 Running 0 135m
pipelines-vote-ui-54fcb9658d-tkqjp 1/1 Running 0 9m29s
polling-triggers-28174084-hrdth 0/1 Completed 0 33s
$ oc logs -f polling-triggers-28174084-hrdth
No changes
// Clone your forked Git repository pipelines-vote-ui, and push an empty commit
$ git clone https://github.com/bysnupy/pipelines-vote-ui.git -b master
$ git commit -m "empty-commit" --allow-empty && git push origin master
// Check if the new pipeline run would be triggered by the polling triggers
$ oc get pod -n pipelines-tutorial
NAME READY STATUS RESTARTS AGE
el-vote-app-68fcd9675f-hdw6j 1/1 Running 0 136m
pipelines-vote-ui-54fcb9658d-tkqjp 1/1 Running 0 9m59s
polling-triggers-28174084-hrdth 0/1 Completed 0 63s
polling-triggers-28174085-s8hzh 0/1 ContainerCreating 0 3s
$ oc logs -f polling-triggers-28174085-s8hzh
{"eventListener":"vote-app","namespace":"pipelines-tutorial","eventListenerUID":"b71d09d8-245f-4126-9c3b-53f399c7c24c","eventID":"169d6f94-29c1-4f65-92fe-1985d6faa3fb"}
$ oc get pod -n pipelines-tutorial
NAME READY STATUS RESTARTS AGE
build-deploy-pipelines-vote-ui-8bxfv-fetch-repository-pod 0/1 Init:0/2 0 23s <--- New pipeline tasks started
el-vote-app-68fcd9675f-hdw6j 1/1 Running 0 136m
pipelines-vote-ui-54fcb9658d-tkqjp 1/1 Running 0 10m
polling-triggers-28174084-hrdth 0/1 Completed 0 89s
polling-triggers-28174085-s8hzh 0/1 Completed 0 29
Wrap up
This post demonstrated polling triggers using CronJob and verified how the process works. You can also customize the logic for your own use cases; for instance, directly running the pipeline without EventListener using the tkn command. Using your own implementation, you can do anything that meets your needs. I hope this information helps deepen your knowledge of the OpenShift Pipeline.
About the author
Daein Park is an OpenShift Software Maintenance Engineer in Japan. He has started his career as a Java Software Engineer, and has experience with various e-commerce projects as an Infrastructure Engineer. He is also an active open source contributor.
Browse by channel
Automation
The latest on IT automation for tech, teams, and environments
Artificial intelligence
Updates on the platforms that free customers to run AI workloads anywhere
Open hybrid cloud
Explore how we build a more flexible future with hybrid cloud
Security
The latest on how we reduce risks across environments and technologies
Edge computing
Updates on the platforms that simplify operations at the edge
Infrastructure
The latest on the world’s leading enterprise Linux platform
Applications
Inside our solutions to the toughest application challenges
Original shows
Entertaining stories from the makers and leaders in enterprise tech
Products
- Red Hat Enterprise Linux
- Red Hat OpenShift
- Red Hat Ansible Automation Platform
- Cloud services
- See all products
Tools
- Training and certification
- My account
- Customer support
- Developer resources
- Find a partner
- Red Hat Ecosystem Catalog
- Red Hat value calculator
- Documentation
Try, buy, & sell
Communicate
About Red Hat
We’re the world’s leading provider of enterprise open source solutions—including Linux, cloud, container, and Kubernetes. We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.
Select a language
Red Hat legal and privacy links
- About Red Hat
- Jobs
- Events
- Locations
- Contact Red Hat
- Red Hat Blog
- Diversity, equity, and inclusion
- Cool Stuff Store
- Red Hat Summit