Introduction

Are you looking to reduce costs in managing your control planes? Are you looking to reduce the burdens of managing multiple control planes? Hosted Control Planes (aka HyperShift) for Red Hat OpenShift makes it possible to host OpenShift tenant clusters at scale in Amazon Web Services (AWS). With each standalone OpenShift cluster, the control plane is scheduled across three machine nodes. However, with Hosted Control Planes, the control plane is scheduled as pods on available worker nodes just like any other workload. This allows many control planes to be created and run on fewer nodes making clusters more affordable while still re-using OpenShift tenancy primitives such as projects, policies, etc. While utilizing hosted control planes can reduce costs, utilizing Arm hosted control planes can also potentially add to those cost savings.

Arm hosted control planes can be installed on existing 64-bit Arm OpenShift Container Platform (OCP) cluster environments allowing you to quickly deploy multiple Arm hosted control planes with 64-bit x86 NodePools. This can all be done on the AWS platform and opens the opportunity to save on costs by utilizing Arm hosted control planes. This blog will examine the benefits and installation steps for installing an Arm hosted control plane.

Please note the current support status of this feature is in Technical Preview (TP) and will be officially supported in a future release.

Benefits

Using Arm hosted control planes, over standard OCP clusters, offers these additional benefits:

• Arm hosted control planes could reduce costs by approximately 20% compared to an equivalent x86 hosted control plane. Cost savings based on per hour cost of m6i.xlarge (x86 based) vs m6g.xlarge (Arm based)
• Reduced maintenance
• Stronger security boundaries between management and workloads
• Run many control planes on fewer nodes making clusters more affordable
• Quicker startup times for control planes
• Makes multicluster management more centralized and easier

More details on hosted cluster benefits can be found in the hosted control plane overview documentation.

Cluster Preparation

OCP 4.13.0 is running as the underlying base for a 64-bit Arm OCP cluster hosted on AWS.

1. Install the multicluster engine operator, 2.2.4 or newer. Instructions for installing this operator can be found here.
2. Next, follow the steps under, 1.7.1.2. Creating the Amazon Web Services S3 bucket and S3 OIDC secret, to set up an AWS S3 bucket and S3 OIDC secret.

   1. After step 1, run the following command:
       

      aws s3api put-bucket-ownership-controls --bucket $BUCKET_NAME --ownership-controls="Rules=[{ObjectOwnership=BucketOwnerPreferred}]"

3. Next, follow the steps under, 1.7.1.6. Enabling the hosted control planes feature, to enable hosted control planes.
4. Download the hypershift cli, 1.7.1.7 Installing the Hosted Control Plane CLI

Demo

The following sections will take you through the steps of:

1. Configuring Environment Variables
2. Creating an Arm Hosted Control Plane and 64-bit x86 Guest Cluster
3. Adding an Additional 64-bit x86 Guest Cluster to the Existing Arm Hosted Control Plane
4. Deleting an Arm Hosted Control Plane

Configuring Environment Variables

First, set up some environment variables needed by the HyperShift command line tool to install the Arm HyperShift Operator and create the hosted control plane cluster. This closely follows step 1, in 1.7.2.3. Deploying a hosted cluster on AWS.

You can obtain the image pull secret from the Red Hat OpenShift Cluster Manager.

REGION=us-east-1
AWS_CREDS=~/.aws/credentials
BASE_DOMAIN=my.base.domain.com
PULL_SECRET=~/all-the-pull-secrets.json
BUCKET_NAME=my-s3-bucket
CLUSTER_NAME=my-hypershift-arm-cluster
NODEPOOL_NAME=my-additional-x86-guest-cluster
NODEPOOL_REPLICAS=2

Set your KUBECONFIG environment variable to point at your OpenShift 64-bit Arm management cluster.

$ hypershift % export KUBECONFIG=/Users/user/aws-oc-nightly/auth/kubeconfig

Creating an Arm Hosted Control Plane and 64-bit x86 Guest Cluster

1) Next, create a hosted control plane with an initial guest cluster with 2 nodes while overriding the standard release image to utilize a multi-architecture release image since the management cluster is 64-bit Arm and the hosted control plane guest clusters will be 64-bit x86. This ensures the right bootstrapping configuration is used when creating the hosted cluster and its guest cluster.

This closely follows step 4, in 1.7.2.3. Deploying a hosted cluster on AWS.

$ hypershift create cluster aws \ 
--name $CLUSTER_NAME \
--node-pool-replicas=$NODEPOOL_REPLICAS \
--base-domain $BASE_DOMAIN \
--pull-secret $PULL_SECRET \
--aws-creds $AWS_CREDS \
--region $REGION \
--release-image quay.io/openshift-release-dev/ocp-release:4.13.0-multi

2) Verify successful guest cluster creation by utilizing the OpenShift command line tool.

$ oc get nodepools --namespace clusters
NAME                                   CLUSTER                     DESIRED NODES   CURRENT NODES   AUTOSCALING   AUTOREPAIR   VERSION       UPDATINGVERSION   UPDATINGCONFIG   MESSAGE
my-hypershift-arm-cluster-us-east-1a   my-hypershift-arm-cluster   2               2               False         False        4.13.0

3) Verify successful creation of the hosted control plane.

$ oc get --namespace clusters hostedclusters
NAME                        VERSION       KUBECONFIG                                   PROGRESS    AVAILABLE   PROGRESSING   MESSAGE
my-hypershift-arm-cluster   4.13.0   my-hypershift-arm-cluster-admin-kubeconfig   Completed   True        False         The hosted control plane is available

4) Once the guest cluster and hosted control plane are successfully created, create a KUBECONFIG to navigate to the new guest cluster and verify the AWS machines are in the guest cluster.

$ hypershift create kubeconfig > temp_kubeconfig
2023/03/21 11:12:32 selected 1 of 1 hostedclusters for the kubeconfig
2023/03/21 11:12:32 adding clusters/my-hypershift-arm-cluster to kubeconfig
2023/03/21 11:12:32 added clusters-my-hypershift-arm-cluster to kubeconfig
2023/03/21 11:12:32 created kubeconfig with 1 contexts
$ hypershift % export KUBECONFIG=/Users/user/hypershift/temp_kubeconfig
$ hypershift % oc get nodes
NAME                           STATUS   ROLES    AGE   VERSION
ip-10-0-142-149.ec2.internal   Ready    worker   10m   v1.26.2+06e8c46
ip-10-0-143-86.ec2.internal    Ready    worker   10m   v1.26.2+06e8c46

Adding an Additional 64-bit x86 Guest Cluster to the Existing Arm Hosted Control Plane

In addition, you can add additional guest clusters, utilizing the HyperShift command line tool, to the hosted control plane as needed.

$ hypershift create nodepool aws \
--cluster-name $CLUSTER_NAME \
--name $NODEPOOL_NAME \
--node-count=$NODEPOOL_REPLICAS
NodePool my-additional-x86-guest-cluster created

Use the OpenShift command line tool command to verify the new guest cluster was initialized successfully.

$ oc get nodepools --namespace clusters
NAME                                   CLUSTER                     DESIRED NODES   CURRENT NODES   AUTOSCALING   AUTOREPAIR   VERSION       UPDATINGVERSION   UPDATINGCONFIG   MESSAGE
my-additional-x86-guest-cluster        my-hypershift-arm-cluster   2               2               False         False        4.13.0
my-hypershift-arm-cluster-us-east-1a   my-hypershift-arm-cluster   2               2               False         False        4.13.0

Deleting an Arm Hosted Control Plane

To delete the Arm hosted control plane, run the following command below. This closely follows step 1, in 1.7.2.6. Destroying a hosted cluster on AWS.

$ hypershift destroy cluster aws \
--name $CLUSTER_NAME \ 
--aws-creds $AWS_CREDS

Summary

We demonstrated the steps needed to install and verify an Arm hosted control plane with an 64-bit x86 tenant cluster. We also demonstrated how to use the HyperShift command line tool to install the Arm HyperShift operator, created an additional 64-bit x86 tenant cluster utilizing the same Arm hosted control plane, verified successful creation of the  64-bit x86 tenant clusters, verified successful creation of the Arm hosted control plane, and deleted the Arm hosted control plane.

Future Work

Future work will expand the multi-architecture CPU capabilities of hosted control planes. The next capability to be added is enabling creation of 64-bit Arm NodePools on AWS. With this capability, you will be able to run 64-bit x86 and 64-bit Arm NodePools from the same hosted control plane (note: you will not be able to mix CPU architectures within the same NodePool).

After the 64-bit Arm NodePools on AWS, we will expand 64-bit Arm hosted control planes and 64-bit Arm NodePools onto other on-prem and cloud providers in the future.