Ask an OpenShift Admin Office Hour - OpenShift on VMware and the vSphere Kubernetes Drivers Operator

OpenShift on VMware vSphere is the most popular deployment type and this week Dean Lewis, from VMware, joined to discuss getting the most out of OpenShift on your vSphere infrastructure. During today’s stream we learned how to configure and best practices for storage, talk about common practices or issues we see with deployments on VMware, and covered integrating with the VMware network stack.

Additionally, VMware recently released the vSphere Kubernetes Driver Operator, a method for deploying the VMware cloud provider and storage drivers. We had a great conversation about where it fits in with OpenShift and when it’s appropriate to use the driver Operator.

We had a great conversation with Dean, including a lot of viewer questions! Please be sure to review the summary of questions below to find each question and where we answered in the stream.

Since we had a (wonderful!) plethora of viewer questions, Dean wasn’t able to get through all of the content we had hoped. As a result, he created a video and blog post to cover that extra material.

As always, please see the list below for additional links to specific topics, questions, and supporting materials for the episode!

If you’re interested in more streaming content, please subscribe to the Red Hat livestreaming calendar to see the upcoming episode topics and to receive any schedule changes. If you have questions or topic suggestions for the Ask an OpenShift Admin Office Hour, please contact us via Discord, Twitter, or come join us live, Wednesdays at 11am EDT / 1500 UTC, on YouTube and Twitch.

Episode 54 recorded stream:

Use this link to jump directly to where we start talking about today’s topic.

This week’s top of mind topics:

• Some follow ups from last week’s stream, we found some additional docs on troubleshooting updates and Operators that may be useful if you encounter issues with the update/upgrade process.
• Something that is a bit less obvious, but you can add credentials for other registries to the global pull secret. This is useful, for example, if you have an enterprise account with Docker Hub that you want to use with all image pulls from there.

Questions answered and topics discussed during the stream:

• Starting off with a viewer question, should you use NSX-T or OpenShiftSDN / OVN-Kubernetes? The answer here is really “it depends”. If you have NSX-T and you’re using its features, for example policy based network security management, then you should absolutely use it with OpenShift. If you don’t have NSX-T, or you have no need to integrate your OpenShift cluster with other things using NSX-T, then there’s no harm in not using it. And, don’t forget you can use Antrea and NSX-T together!
• Also from a viewer, are there any special integrations between VMware and OpenShift with regard to SR-IOV and the Operator? Unfortunately not. The Operator does not integrate with the hypervisor to request or configure those devices, rather it works at the RHCOS level to configure the devices when they’re available.
• What about OpenShift Virtualization and VMware? This viewer question cuts right to the core of many other questions we get around OpenShift Virtualization. The two technologies are different and don’t really overlap. OpenShift Virtualization enables VMs to be hosted as and run as Pods in an OpenShift cluster. This is different from an OpenShift cluster deployed to VMware. We talk about the different use cases here during the stream.
• Is nested networking supported with OpenShift on VMware? The question was completely clear here, so we did our best - ultimately, there’s nothing wrong with using OpenShiftSDN or OVN-Kubernetes on top of and/or alongside NSX-T. But, you also have the option of using NSX-T natively with OpenShift.
• Dean uses a slide here in the stream to illustrate the state of OpenShift and VMware integration as well as set the stage for some further discussions.
  
  

• Dean highlights that vMotion is supported with OpenShift, but storage vMotion is not. The documentation highlights this as well. Storage vMotion of, in particular, the VMDKs backing PVs will result in the storage provisioner - both in-tree and CSI - losing the link between the Kubernetes object and the VMware object. This makes it impossible for the disk to be mounted and used by the Pod(s).
• Are there any performance impacts to running containerized applications in OpenShift on VMware? There is effectively no performance impact to running the application containerized on VMware - including using OpenShift as the Kubernetes platform - vs non-containerized on VMware.
• Dean does a great job highlighting how OpenShift and vSphere work together to provide an amazing experience here in the stream, discussing how vSphere high availability (HA) returns an OpenShift node on a failed hardware node to service faster than OpenShift / Kubernetes declares it unreachable. This means that workload can be rescheduled in less than 60 seconds instead of 5+ minutes.
• Is it possible to use OpenShift with VMware on AWS (VMC) and VMware on Azure? With VMC, yes! This is tested and documented by Red Hat, it’s a fully supported installation platform. For VMware on Azure, it would fall into the untested infrastructure category - so, still supported, but with some constraints.
• How do OpenShift availability zone concepts map to vSphere availability concepts? Unfortunately, OpenShift does not have zone awareness through the cloud provider, but you can - with a UPI or non-integrated install - manually configure tags on the OpenShift nodes to use for Pod (anti)affinity rules. However, OpenShift clusters spanning multiple vCenters are not supported.
• Dean talks about automation integration between VMware and OpenShift here. Some really interesting things are possible between the two platforms, including using some example vRealize Automation to deploy OpenShift clusters.
• Digging into integrating the VMware network stack with OpenShift, Dean does a deep dive and demo of Antrea and NSX-T here. This includes some really powerful multi-cluster, policy-based security rules that are able to be applied by the network administrator.
• Is NSX-V supported with NCP and OpenShift? No, only NSX-T.
• Does Istio / Service Mesh work with Antrea and/or NSX-T? Or does Antrea supercede the capabilities of a Service Mesh? They have different use cases. Service Mesh, in particular capabilities like Jaeger, offer visibility into application API calls and other higher level data, whereas Antrea and NSX-T operate at the network level to provide security rules.
• One of our astute viewers asked “is NCP is still useful with the modern capabilities of Antrea and NSX-T?” Dean does a great job highlighting the different capabilities of the offerings, including adding external metal nodes with Antrea, but also having to bring an external load balancer with Antrea, whereas NCP has an integrated offering.
• Does the OpenShift Assisted Installer work with VMware? Yes, currently with no platform integration. However, platform integration is on the roadmap!