This post was written with the help of Andy Daniel at Nutanix

Red Hat OpenShift on Nutanix AOS with AHV

Starting with the Red Hat OpenShift 4.11 release, we are extending the infrastructure provider integration experience to a new platform, Nutanix AOS.

This allows Nutanix customers to fully automate the Red Hat OpenShift deployment using installer-provisioned infrastructure on Nutanix AOS using the Nutanix hypervisor, AHV. Red Hat is planning to support deploying Red Hat OpenShift on user-provisioned infrastructure in a future release using Assisted Installer.

Nutanix hyperconverged infrastructure solution

With the lowest TCO of any datacenter platform, Nutanix enables simple deployment, management and scaling of OpenShift workloads. By bringing a cloud-like experience and consumption model to the datacenter or edge, the highly automated platform delivers new services in minutes and minimizes operational overhead. The complete stack, with built-in data services like a full featured CSI for file and block storage, native S3 object storage, and an integrated multi-vendor DBaaS, is a natural fit for customers ready to migrate business-critical, persistent workloads to Kubernetes. Now with full support for OpenShift installer-provisioned infrastructure when combined with the Nutanix AHV hypervisor, customers have the choice of not only underlying hardware but hypervisor as well. Simply put, when combined, Nutanix AOS and Red Hat OpenShift represent the best-in-class, full-stack solution for VMs, DevOps and container management.  

Deploying OpenShift on Nutanix AOS using Installer-Provisioned Infrastructure (IPI)

Detailed instructions on how to deploy OpenShift on Nutanix AOS are available here. This post provides a high-level overview of the required steps.

To get started, you need an account in the Nutanix AOS environment with administrative privileges to read and create the required resources. You will also need to create some networking resources before you install the OpenShift Container Platform cluster, such as VIP addresses for the API and Ingress services as well as the corresponding DNS records to resolve these services.

With these prerequisites met, you can start the first phase of the installation by generating the install config. 

$ ./openshift-install create install-config --dir mak-nutanix
? SSH Public Key /Users/mak/.ssh/
? Platform nutanix
? Prism Central
? Port 9440
? Username ****************
? Password [? for help] ****************
INFO Connecting to Prism Central
? Prism Element Development-LTS
? Subnet uvm_network1
? Virtual IP Address for API
? Virtual IP Address for Ingress
? Base Domain
? Cluster Name mak-test
? Pull Secret [? for help] ****************
INFO Install-Config created in: mak-nutanix

Installing the cluster for Nutanix AOS requires that the Cloud Credential Operator (CCO) operate in manual mode. While the installation program configures the CCO for manual mode, you must specify the identity and access management secrets. We have some specific actions to extract the list of CredentialsRequest custom resources and generate the credential files.

$ ./ccoctl nutanix create-shared-secrets

ccoctl nutanix create-shared-secrets [flags]

        --credentials-requests-dir string          Directory containing files of CredentialsRequests (can be created by running 'oc adm release extract --credentials-requests --cloud=nutanix' against an OpenShift release image)
        --credentials-source-filepath string   The filepath of the nutanix credentials data. If not specified, will use the default path ~/.nutanix/credentials
        --enable-tech-preview                      Opt into processing CredentialsRequests marked as tech-preview
-h, --help                                     help for create-shared-secrets
        --output-dir string                        Directory to place generated files (defaults to current directory)

Next, we can proceed with generating the initial manifests and copying the generated credential files to the target manifests directory.

$ ./openshift-install create manifests --dir mak-nutanix
INFO Consuming Install Config from target directory
INFO Manifests created in: mak-nutanix/manifests and mak-nutanix/openshift

With the manifests updated, you can proceed with the cluster installation. At this point, the rest of the installation process looks the same as any other provider.

$ ./openshift-install create cluster --dir mak-nutanix --log-level=info
time="2022-07-29T21:15:50Z" level=info msg="Connecting to Prism Central"
time="2022-07-29T21:16:49Z" level=info msg="Install-Config created in: ."
time="2022-07-29T21:19:34Z" level=info msg="Consuming Install Config from target directory"
time="2022-07-29T21:19:35Z" level=info msg="Manifests created in: manifests and openshift"
time="2022-07-29T21:22:00Z" level=info msg="Consuming OpenShift Install (Manifests) from target directory"
time="2022-07-29T21:22:00Z" level=info msg="Consuming Worker Machines from target directory"
time="2022-07-29T21:22:00Z" level=info msg="Consuming Master Machines from target directory"
time="2022-07-29T21:22:00Z" level=info msg="Consuming Common Manifests from target directory"
time="2022-07-29T21:22:00Z" level=info msg="Consuming Openshift Manifests from target directory"
time="2022-07-29T21:22:00Z" level=info msg="Ignition-Configs created in: . and auth"
time="2022-07-29T21:22:22Z" level=info msg="Consuming Master Ignition Config from target directory"
time="2022-07-29T21:22:22Z" level=info msg="Consuming Bootstrap Ignition Config from target directory"
time="2022-07-29T21:22:22Z" level=info msg="Consuming Worker Ignition Config from target directory"
time="2022-07-29T21:22:22Z" level=info msg="Creating infrastructure resources..."
time="2022-07-29T21:25:33Z" level=info msg="Waiting up to 20m0s (until 9:45PM) for the Kubernetes API at"
time="2022-07-29T21:26:26Z" level=info msg="API v1.24.0+9546431 up"
time="2022-07-29T21:26:26Z" level=info msg="Waiting up to 30m0s (until 9:56PM) for bootstrapping to complete..."
time="2022-07-29T21:42:33Z" level=info msg="Destroying the bootstrap resources..."
time="2022-07-29T21:43:10Z" level=info msg="Waiting up to 40m0s (until 10:23PM) for the cluster at to initialize..."
time="2022-07-29T21:52:53Z" level=info msg="Waiting up to 10m0s (until 10:02PM) for the openshift-console route to be created..."
time="2022-07-29T21:52:53Z" level=info msg="Install complete!"
time="2022-07-29T21:52:53Z" level=info msg="Access the OpenShift web-console here:"
time="2022-07-29T21:52:53Z" level=info msg="Time elapsed: 30m32s

Once the cluster has been deployed completely, you can check the Nutanix Prism Central UI to see all the resources the installer has created for you, like the VMs that will conform the OpenShift Cluster as we can see below.

You can also view and manage your deployed cluster in the Red Hat OpenShift Console.

 Including the Machines that represents the VMs we have just seen in Prism Central, where you can scale up and down your cluster automatically leveraging the integration between OpenShift and the Nutanix platform.


Check out episode 79 of our “Ask an OpenShift Admin” series titled OpenShift and Nutanix: IPI, CSI, and more!” to learn more about the Installer Provisioned Infrastructure (IPI) workflow and deploying OpenShift 4.11 on Nutanix Cloud Platform. In this episode Andy Daniel, Principal Product Manager for Nutanix, discusses the road to OpenShift on Nutanix AHV. You will also see a demonstration of the installation process and explore further integration between the two platforms. You will learn why is IPI/UPI support “important” and why we spent one year developing the capability.

This new provider integration enables existing or new Nutanix customers to explore the usage of OpenShift on Nutanix AOS and take advantage of the benefits offered by the platform.

Red Hat and Nutanix will continue working together in this integration to add more features to OpenShift deployments on this new platform in future releases.

We are really excited to see how customers take advantage of this new deployment option and look forward to working with them on their hybrid cloud journey.

Please leave a comment or send us feedback either through your usual Red Hat contacts or as an issue on OpenShift on Github.