Pipelines with Jenkins 2 on OpenShift

Jenkins 2 is finally released. Jenkins 2 brings lots of improvements such as built-in support for delivery pipeline as code, a brand new setup experience and other UI improvements all while maintaining total backwards compatibility with existing Jenkins installations.

While the Pipeline (previously named Workflow) plugin was available for installation on Jenkins 1.x, the built-in support for deployment pipelines as a first-class entity is interesting since it allows defining more complex pipelines through introducing a Groovy-based domain-specific language (DSL) that helps Jenkins users to model their software deployment pipelines as code. Not only is DSL a more flexible way to define pipelines but also the DSL file can be checked-in in a source repository and version-controlled along with the rest of their project's source code, promoting the configuration-as-code principle.

Jenkins 2.x backward compatibility allows it to be a drop-in replacement for Jenkins 1.x which enables users to import their current job configurations with no efforts.

OpenShift container application platform ships Jenkins 1.x container images with the platform and integrates with it features for building and deploying applications from Jenkins on to OpenShift. While OpenShift will include new versions of Jenkins in future releases, in this blog post we look at how to deploy and use Jenkins 2.x on the current version of OpenShift.

Deploying Jenkins 2 on OpenShift

OpenShift can pull and deploy docker images from upstream registries (e.g. Docker Hub) with a simple new-app command. It will first search the internal and local image repository and then check for the image in the configured upstream repositories:

$ oc new-project ci

$ oc new-app library/jenkins:2.0

$ oc expose svc jenkins

OpenShift allows adding persistent storage to containers so that the data on the filesystem survives when containers are restarted. Instead of deploying the Jenkins image directly, you can use the Jenkins persistent template in order to deploy Jenkins and automatically configure the persistent storage as well as a route for accessing Jenkins web console:

$ oc new-project ci

$ oc new-app https://raw.githubusercontent.com/siamaksade/openshift-builds/master/jenkins-template/jenkins-persistent-template.yaml

Installing Jenkins 2

Jenkins has revamped the installation process to assist with setting up accounts and picking relevant plugins. Pointing your browser at the Jenkins url starts the installation wizard. If you are using the free Red Hat Container Development Kit (CDK), the Jenkins url would be: http://jenkins-ci.rhel-cdk.10.1.2.2.xip.io

In order to ensure security in the installation process, Jenkins requires the admin password. The password is printed in the logs and also is stored in a file on the Jenkins pod at /var/jenkins_home/secrets/initialAdminPassword.
You can view the logs in the Web Console or use the oc logs command to copy the password:

Alternatively you can get the password from the pod by looking at the content of the password file:

$ oc get pods

$ oc exec jenkins-1-tw36y cat /var/jenkins_home/secrets/initialAdminPassword

1df5e3f646a94576afbd1db9e1c7f7f2

Jenkins 2 simplifies installation of plugins by integrating that into the installation process. If you want to cherry-pick a list of plugins choose Select plugins to install otherwise choose Install suggested plugins to allow Jenkins install the most common plugins. You can always add more plugins to Jenkins after installation is complete.

After plugin installation, create an admin user in the installation wizard. Since a Maven-based project is used in this post, you should define a Maven tool in Jenkins configuration. Go to Manage Jenkins → Global Tool Configuration and Add Maven. Take note of the Maven name since you need to refer to it when creating deployment pipelines. M3 is used as the Maven name in the rest of this post.

Pipelines

Pipeline is a new type of built-in job in Jenkins 2. When creating a new job, you can choose Pipeline among other job types to create a delivery pipeline.

Job configuration includes a DSL editor for defining the pipeline using the Pipeline DSL. The complete DSL reference is available here. Jenkins also includes a DSL generator to help you learn the Groovy code used in the DSL. Make sure Script Generator is checked to enable this utility. Pick a step you are interested in from the list, configure it, click Generate Groovy, and you will see a Groovy statement that would call the step with that configuration. You may copy and paste the statement into your script.

The following is a simple pipeline for checking out, building and testing the code with Maven.

The DSL editor works fine for simpler scenarios however complex pipelines would be too cumbersome to manage in the editor. Therefore you also have the option of writing pipeline scripts called Jenkinsfile with the editor of your choice or IDE (auto-complete support can be configured in IntelliJ IDEA) and storing them in your version control system. Jenkins can load those scripts using the Pipeline Script from SCM option. This approach is the recommended way to build pipelines since it allows updating the pipeline automatically when the Jenkinsfile in the source repository is updated.

When starting a pipeline manually or through a trigger, the job page visualizes the pipeline as it progresses through the stages.

Deploying Applications to OpenShift

The OpenShift Pipeline Plugin can be used to orchestrate actions on OpenShift directly from the Pipelines DSL. Doing so allows integrating deployment of the application in various environments (DEV, TEST, etc) as stages in the pipeline and define the pipeline flow accordingly. A use-case for that is creating an on-demand TEST environment on OpenShift and run the system integration tests against the deployed application directly from the pipeline.

Jenkins 2 is backwards compatible with Jenkins 1.x and therefore the OpenShift Pipeline Plugin which is developed for Jenkins 1.x also works with Jenkins 2 and can be installed through Manage Jenkins → Manage Plugins.

Check out this video and blog post to learn how to use the OpenShift Pipeline Plugin for deploying applications to OpenShift.

The OpenShift Pipeline Plugin is also pre-installed in the OpenShift Jenkins Image on DockerHub (docker.io/openshift/jenkins-1-centos7), although this Jenkins image is based on Jenkins 1.x.

The OpenShift Pipeline Plugin will extend the base pipeline DSL in future releases in order to be used directly within the pipeline definition. Nevertheless, the current release of the OpenShift Pipeline Plugin (v1.0.10) is compatible with Jenkins 2.x pipelines and can be used within the pipeline through instantiating and calling the build steps:

def builder = new com.openshift.jenkins.plugins.pipeline.OpenShiftBuilder(

                                   apiURL,

                                   buildConfig, 

                                   project,  

                                   authToken,

                                   verbose,  

                                   commitID,

                                   buildName,

                                   showBuildLogs,  

                                   checkForTriggeredDeployments,  

                                   waitTime)

The builder step has sensible default values for most arguments.  The parameters above in bold should be specified, whereas the others can have “null” or “”. Specify the project and build config name in order to trigger build for the application, and specify “true” or “false” for whether you want verbose logging, dumping of the build logs upon completion, or if you want to enable post build check to see if deployments triggered by the build’s output image changing. The following build steps are available:

• OpenShiftScaler: Scale a deployment up or down
• OpenShiftDeployer: Trigger a deployment
• OpenShiftServiceVerifier: Verify a service exists
• OpenShiftImageTagger: Tag an image
• OpenShiftDeploymentVerifier: Verify a successful deployment
• OpenShiftBuildVerifier: Verify a successful build
• OpenShiftCreator: Create resource(s)
• OpenShiftDeleterJsonYaml: Delete resource(s) via JSON/YAML
• OpenShiftDeleterList: Delete a list of resource(s)
• OpenShiftDeleterLabels: Delete resources based on labels

For more details on the above build steps consult the plugin’s documentation.

Create a new project in OpenShift using the JBoss 6.4 S2I Template and deploy the Ticket Monster application which will be used to trigger builds through the pipeline. Source-to-Image (S2I) is a build process that produces ready-to-run Docker images by building and injecting the application code into an existing S2I-enabled Docker image. Ticket Monster is a Java EE application used as the reference application on JBoss EAP. Everytime the code is successfully built through the pipeline, we want to trigger a new deployment of the Ticket Monster application in this project.

$ oc new-project demo

$ oc new-app eap64-basic-s2i \

--param=APPLICATION_NAME=ticket-monster \

--param=SOURCE_REPOSITORY_URL=https://github.com/jboss-developer/ticket-monster.git \

--param=SOURCE_REPOSITORY_REF=2.7.0.Final \

--param=CONTEXT_DIR=demo

Now we can modify the Pipeline to deploy the application to OpenShift. Configure the pipeline job and use the following Groovy script for the pipeline:

node {

stage 'Checkout'

git branch: '2.7.0.Final', url: 'https://github.com/jboss-developer/ticket-monster.git'
// ** NOTE: This 'M3' maven tool must be configured in the global configuration.
 def mvnHome = tool 'M3'
stage 'Build'
 sh "${mvnHome}/bin/mvn -f demo/pom.xml clean install"
stage 'Deploy'
 def builder = new com.openshift.jenkins.plugins.pipeline.OpenShiftBuilder("", "ticket-monster", "demo", "", "", "", "", "true", "", "")
 step builder
 }
 

Make sure to uncheck Use Groovy sandbox checkbox to disable the sandbox. The Deploy stage uses the Builder step from OpenShift Pipeline Plugin to start a build and deployment on OpenShift. Since Jenkins performs actions on the demo project, you need to grant the service account used by Jenkins for invoking OpenShift APIs, enough permissions to perform those actions on the demo project.

$ oc policy add-role-to-user edit system:systemaccount:ci:default -n demo

Since Jenkins is running in a container on OpenShift, the OpenShift Pipeline Plugin is able to discover the OpenShift API endpoint and use the service account’s credentials, which are available inside the container, to authenticate against the API.

Conclusion

Jenkins 2 provides a convenient approach for building deployment pipelines by bringing in the Pipeline plugin as a first-class citizen. Pipeline’s Groovy DSL is a more flexible and powerful way of building pipelines compared to other pipeline plugins available for Jenkins particularly when it comes to running parallel jobs and branching. The SCM integration for pipeline definitions (Jenkinsfile) enables treating them as code and even dynamically create new pipelines as new Jenkinsfiles appear in the source repository.
Jenkins 2 can be deployed on-demand as a container on OpenShift and can orchestrate actions on OpenShift through the OpenShift Pipeline Plugin. Future versions of OpenShift will provide a tighter integration with Jenkins and allow automatic creation of pipelines on Jenkins when new container applications are created. Furthermore, pipelines will be visualized in OpenShift Web Console so that users can have a complete view of the deployment pipeline in a single place.