Introducing Red Hat build of Karpenter

Achieving infrastructure efficiency and controlling compute costs is a continuous effort. While traditional machine pools are effective for steady workloads, scaling diverse applications often requires balancing cloud utilization with the effort spent manually right-sizing instances, all while providing continuous node availability when compute demands spike. To help achieve this balance, we are introducing the Red Hat build of Karpenter, based on the upstream Karpenter project. With the release of Red Hat OpenShift 4.22, Red Hat build of Karpenter can be enabled on Red Hat OpenShift Service on AWS with hosted control planes.

What is Karpenter?

Karpenter is a high-performance, Kubernetes-native node autoscaler. Instead of scaling fixed pools of identical machines, it looks at the collective resource requirements of your pending pods and provisions the right compute resources just-in-time, then continuously consolidates the cluster to reduce your infrastructure costs. The result is compute that is right-sized automatically, around the clock, with no manual intervention.

What is the Red Hat build of Karpenter?

Red Hat build of Karpenter brings workload-aware, just-in-time node provisioning to Red Hat OpenShift Service on AWS with hosted control planes. Instead of managing static machine pools with pre-defined instance types, Karpenter evaluates the exact CPU, memory, and scheduling constraints of pending pods and provisions the optimal EC2 instance automatically, and then consolidates underutilized nodes when they are no longer needed.

Controllers hosted in the control plane

The Karpenter controllers run as part of the hosted control plane, not on your worker nodes. There are no extra pods to manage, no added compute overhead, and no resource contention between Karpenter and your applications.

Enable on existing clusters

Karpenter can be turned on for existing clusters once they are upgraded to OpenShift 4.22. There is no need to recreate your cluster to adopt it.

Independent upgrades

The Red Hat build of Karpenter allows you to upgrade the worker nodes either along with or independently of the hosted control plane on the schedule that fits your requirements. 

Coexistence with Cluster Autoscaler

Karpenter and the Cluster Autoscaler can all run in the same cluster. You can migrate gradually from self-managed to Karpenter-managed NodePools.

Capacity Reservations and Capacity Blocks for ML

Karpenter works with reserved capacity, including Amazon EC2 On-Demand Capacity Reservations (ODCRs) and Capacity Blocks for ML. This is critical for machine learning training and for workloads with regulatory requirements that demand reserved compute. Also, Karpenter can dynamically choose between on-demand and Capacity Reservation market types based on utilization of the reservation.

Kubelet configuration and node tuning

For advanced use cases, you can apply kubelet configurations and TuneD profiles to meet the needs of diverse workloads.

Security compliance

The Red Hat build of Karpenter is designed to slot into your existing security posture rather than work around it. Red Hat provides the compliance foundation, including FIPS, SOC 2, and FedRAMP, while Karpenter respects all Kubernetes scheduling constraints, taints, and tolerations. 

Cost savings 

This shift to intelligent node management helps your organization achieve cost savings in four key ways:

• Automatic right-sizing: Instead of using fixed groups of identical machines, Karpenter looks at exactly what your applications need right now and chooses the most cost-effective size to fit them. This means you stop paying for unused computing power.
• Just-in-time scaling: Nodes spin up instantly when your application traffic spikes to keep things running smoothly, and they disappear the moment demand drops so you aren't paying for idle servers.
• Utilizing Spot Instance: Karpenter dynamically compares prevailing market rates for various Spot instances with On-Demand pricing and selects the least expensive one(s) that still meets your workload requirements.  
• Maximizing existing AWS commitments: Karpenter automatically prioritizes your pre-purchased AWS assets—such as On-Demand Capacity Reservations (ODCRs) and Capacity Blocks for ML. It utilizes your discounted capacity first before falling back to buying brand-new compute.
• Reducing operational overhead: By automating the sizing, monitoring, and day-to-day optimization of your cluster infrastructure, Karpenter eliminates manual node-pool management so platform teams can focus on higher-priority work.

Best Practices for smarter autoscaling 

As you adopt Red Hat build of Karpenter on your ROSA clusters, keep these 5 recommendations in mind:

1. Keep your Amazon EC2 instance requirements flexible: By allowing a diverse range of EC2 instance types and sizes,  Karpenter can dynamically select the most cost-effective compute for your ROSA clusters. Review supported ROSA EC2 instance types and define your criteria as broadly as possible for maximum cost savings.
2. Align cluster defaults with your operational needs: Karpenter-provisioned nodes are set to expire and initiate an automatic drain after 30 days, so you will want to ensure your applications are ready for this regular rotation. Additionally, if your workloads require highly stable environments that cannot tolerate the sudden interruptions of Spot instances, you should proactively update your NodePool configuration to restrict provisioning to On-Demand instances exclusively. 
3. Rate limiting disruptions: While Karpenter continuously strives to scale, consolidate, and provision nodes to save costs, you can still rate limit the disruptions according to your application and operational requirements. You can achieve this by setting node disruption budgets to protect critical workloads and prevent aggressive node replacement. Additionally, you can define a clear consolidation policy to inform Karpenter whether it should actively consolidate any underutilized nodes or restrict its actions strictly to nodes that are completely empty. 
4. Observe with in-cluster monitoring, logging, and alerting: ROSA exposes prometheus-format metrics in the in-cluster monitoring stack, allowing you to use the dashboards on the OpenShift console for tracking provisioning latency, node utilization and capacity trends. You should configure alerting rules and destinations as needed to get notified for important events. If needed, you can enable logging to receive Karpenter logs from the hosted control plane and route them to a destination of your choice, such as Amazon S3 or CloudWatch.  
5. Validate in non-production first: You should always deploy and validate your NodePool configurations, disruption policies in a staging or non-production environment first. This allows you to safely observe how your specific applications handle node drains, scaling spikes, and rescheduling before you push those changes to production. 

Conclusion

Red Hat build of Karpenter gives platform teams a powerful and trusted option for managing cluster capacity. To learn more:

• Visit the Red Hat OpenShift on AWS web page to see how the managed service fits your environment.
• Read the documentation to get started with the Red Hat build of Karpenter on your cluster.
• Read the Karpenter documentation for a deep dive into the upstream project. 
• Explore the Karpenter interactive walkthrough