A cloud architect's guide to resource and capacity management

One of a cloud architect's responsibilities is to establish and maintain a cloud's capacity and availability at a justifiable cost and with efficient resource usage. This involves managing resources and capacity for the infrastructure (compute, storage, networking), software, and people.

In the first article in this series, we highlighted various aspects of cloud architecture; in the second article, we described self-services delivery; and in the third article, we explored operations. Keeping the platform stable and available requires a disciplined approach to resource and capacity management. The right approach seeks to balance optimizing the platform for cost, availability, and performance.

As we wrote in the first article, Capability Maturity Model Integration (CMMI) provides a framework for the maturity of the processes that combine the people, procedures, and tools to deliver capabilities. The CMMI process areas most relevant to resource and capacity management are:

• Requirements Management (REQM)
• Measurement and Analysis (MA)
• Capacity and Availability Management (CAM)
• Service Continuity (SCON)

Mature teams understand these requirements at different levels. They can measure and predict future resource requirements based on historical data and visibility into the potential future workload.

Establish a strategy

Cloud platform teams need to establish and maintain a capacity and availability management strategy that accounts for quickly responding to capability changes.

They typically start with guardrails to control capacity demand as users consume services. They also use showback and chargeback capabilities, predictive analytics, and automation to push for ways to expand the cloud continually and as required.

The strategy should include resource overcommitment policies that apply to different environments and service-level objectives (SLOs). Ideally (and depending on the service supported), the team should avoid operating production services on overcommitted hardware configurations.

[ Don't miss 6 must-read books for aspiring cloud architects. ]

Know how and when to expand

Capacity management requires a clear understanding of procurement processes and funding models. The cloud team needs to know how long it takes to expand the cloud—taking into account all organizational levels and the level of project commitment required—before they can expand the platform. Enhanced project visibility improves the ability to predict resource requirements. Teams should resist the temptation to introduce or increase existing overcommitments in production environments to satisfy a new application or a customer due to a lack of project planning.

Consider the efficiency of the workload

Kubernetes-based clouds are driving a new set of capabilities such as scale to zero and cloud-optimized application runtimes that can significantly impact resource usage when deployed at scale. Two examples include:

• Tekton Pipelines: These provide serverless Kubernetes-native pipelines that run only when and where needed across cloud platforms.
• Quarkus: This is a Kubernetes-native Java stack that uses a fraction of memory and CPU resources compared to traditional Java runtimes. For example, a large European telco saw a 50% reduction in memory requirements and saved on infrastructure costs when swapping out Spring Boot for Quarkus.

Conclusion

Effective, efficient resource and capacity management are among the most challenging components of running a private cloud environment compared to a public cloud. Keeping track of usage metrics and having a forward-looking view combined with a clear description of cost per compute, memory, and storage unit will help your team anticipate and justify platform expansion when you need it.