How Centris built a reliable, repeatable, and resilient deployment process

Do you remember the ‘Telephone Game’ from your childhood? Especially the version where each player drew something on the next player’s back and so on. Each round produced unique and unpredictable results.

In IT, such effects are not desired when running deployments. You want stable, repeatable and predictable results from running software.

In the following blog, join Red Hat customer Centris and Red Hat partner ipt on their journey to transform a ‘telephone game’ deployment process into a reliable pipeline. As ipt, we will show why it is important to analyze the existing process, how we designed a new process, and the technologies we used to implement the future pipeline. With the introduction of the new order process to their customers, Centris successfully completed their first journey and can now confidently tackle further improvements and challenges.  

A new journey: From ‘Telephone Game’ to ‘Full Confidence’

Centris, a leading provider of IT platforms and software products for health insurance companies in Switzerland, embarked on a journey to transform its historically grown, complex deployment process into a modern, fully automated product ordering application. On this journey and with the support of ipt, a local Red Hat partner and service provider, Centris introduced new cloud-native technologies such as Tekton pipelines and new processes such as GitOps with ArgoCD on Red Hat OpenShift Container Platform to simplify and streamline the deployment of platforms and products.

The journey begins: Analyzing the existing deployment process

As a first step in this journey, we analyzed the existing deployment process by reading documentation and conducting interviews with all involved stakeholders. The result was a diagram of the current process.

The key findings in the diagram were:

• Multiple duplication of deployment manifests with custom templating mechanisms
• Multiple steps in the process with media breaks and manual copy and paste
• Involvement of non-DevOps teams that slow down the process
• No pre-deployment environment checks and no post-deployment smoke tests for verification

With these key areas for improvement, the next steps in the journey could be addressed.

The journey continues: Outlining a new deployment process

In designing the new process, we followed DevOps principles, including a shift-left approach. This means eliminating media breaks and manual copy and paste but also keeping the involvement of non-DevOps teams to an absolute minimum.

To achieve these goals, we decided to implement new technologies:

• A machine-readable product and platform description format to replace the existing Excel and Confluence-based description
• Replace Ansible playbooks run by an Ops team (non-product team) with Tekton pipelines run by the product team
• Replace all custom manifests with Helm Charts and consolidate all values including secrets by introducing Sealed Secrets
• Introduce ArgoCD for a GitOps approach to platform and product deployments

These changes allowed us to begin implementation as the biggest step in the journey.

The journey gets physical: Implementing the new deployment process

The first thing we had to implement was the required multi-tenancy capability of the new deployment process. This means multiple ArgoCD instances with all the configuration for each tenant.

Using a master ArgoCD instance to bootstrap the entire multi-tenant infrastructure did the trick, allowing a single command to set everything up.

While setting up the infrastructure, new components were created to handle the new product and platform description. With everything running on Red Hat OpenShift, we decided to write our microservices in Quarkus because it provided the best developer experience.

In addition, the new Tekton pipelines were designed and implemented. There were two pipelines. The first validated the environment to ensure the installability of the platforms and products, and also generated ArgoCD resources from the new platform and product descriptions, which were automatically applied once they were pushed to the Git repository. The second pipeline performed smoke tests, validating the current deployment to ensure everything was up, running and stable.

As a side project, a small portal was created to provide all tenants with an overview of the installed platforms and products, as well as the API documentation for each component in the products.

The journey ends: Introducing the new process to customers

Centris provides IT platforms and software products to customers, so the first ‘real life’ deployment outside of the development environment was the milestone to achieve.

And when the day came, everyone on the project team was on tenterhooks as the new deployment process ran for the first time. The first pipeline, creating ArgoCD resources after validating all environments and configurations, passed without a hitch. Checking the differences in the Red Hat OpenShift manifests in the ArgoCD console proved the success of this first part.

The second pipeline was more interesting because most of the project was focused on improving all kinds of tests, such as smoke tests or data tests, to increase the quality of the platforms and products. The moment when all test cases turned green was the most satisfying.

Recap the journey: What we accomplished and our key takeaways

Remember the ‘Telephone Game’ from the beginning? For Centris, this is history. The journey has been exciting, sometimes hard, but mostly fun. It was also very educational, and for us those were the most important takeaways from this trip:

• Eliminate media breaks and middlemen: Use the same manifests and configurations from development to production with a single source of truth to avoid configuration drift.
• Implement end-to-end automation: Use pipelines and a control loop to ensure reproducibility and eliminate manual changes.
• Ensure environmental compatibility and deployment stability: Use dependency management and consistent versioning for all components to always check for compatibility.

The new process gives customers full confidence in the quality of the platform and products. Time to market has also been dramatically reduced, with improved automation and stability giving developers full confidence in their process, allowing for more regular releases in the future.

The next journey: Improve and expand

With the success of the first journey, our adventure at Centris is not over yet. We still have a lot of work to do. We need to extend the pipeline capabilities to support more testing technologies, replace the custom developer portal with Red Hat Developer Hub, and add more platforms, products, and components to the new pipeline.

The new process will serve as a role model for future developments at Centris, helping to win more customers and deliver superior products and services.