Ask an OpenShift Admin Office Hour - Kubernetes API Deprecations

Are your Operators ready for the release of Kubernetes 1.22? It's a good idea to double-check because several APIs that have previously been marked as deprecated will be removed and are no longer available in 1.22. 

To help us discuss this change Rob Szumski, Senior Manager, Openshift Product Management, joins us to discuss the details and ensure we avoid removed API versions. We also take a look at the steps needed to upgrade to newer APIs because Operator projects using these API versions will not work on Kubernetes 1.22, or any cluster vendor using Kubernetes version 1.22, such as OpenShift 4.9+.

You can find the slides used during the stream here.

As always, please see the list below for additional links to specific topics, questions, and supporting materials for the episode!

If you’re interested in more streaming content, please subscribe to the Red Hat livestreaming calendar to see the upcoming episode topics and to receive any schedule changes. If you have questions or topic suggestions for the Ask an OpenShift Admin Office Hour, please contact us via Discord, Twitter, or come join us live, Wednesdays at 11am EDT / 1500 UTC, on YouTube and Twitch.

Episode 44 recorded stream:

Use this link to jump directly to where we start talking about today’s topic. 

This week’s top of mind topics:

• The stable upgrade edge for OpenShift 4.7 to OpenShift 4.8 has been added! If you’ve been waiting for stable, now is the time to update with confidence! We also talked about some statistics around this process during the segment, so you may want to give it a listen.
• Two CVEs were announced for Kubernetes last week: CVE-2020-8561 and CVE-2021-25741. I recommend following the CVE pages (you need to be logged in to see the follow button) to be informed of updates when they happen.
• We talked about Windows nodes and Windows Machine Config Operator version 3 last week, however it was not released as “Generally Available”. This week we’re happy to announce it’s been fully released!

Questions answered and topics discussed during the stream:

• We start with an overview of the Kubernetes API policy and, for Operators, which API removals are going to affect the most deployments.
• It’s very important to understand that there are a large number of APIs being removed with Kuberntes 1.22, some have been deprecated since Kubernetes 1.6, so it’s possible that you may have applications that are using the APIs too. Be sure to use the tools available - which we discuss during the stream - to identify which APIs are in use and by who.
• One of the most commonly used API endpoints for Operators is the CustomResourceDefinition (CRD), which will be moving to a v1 spec and dropping the beta. If you have created any custom Operators for your organization, please make sure to check and update this one if needed.
• As an administrator, what do the API versions mean and what impact does it have? As an API matures, the version changes. Alpha, beta, etc. are indicators of how much change is expected to happen in the future.
• Does the version of an API, e.g. alpha or beta, affect the support status from Red Hat? It depends. Some Operators and feature shipped in OpenShift use beta APIs which are fully supported by Red Hat. Other times, beta APIs in upstream Kubernetes are unsupported - and are often disabled in OpenShift as a result.
• Alert manager will trigger alarms when an API that will be removed is in use. This gives a very quick and visible indicator that something needs to be updated in the cluster.
• You can check current and “live” API usage in the cluster with the ApiRequestCounts API object. This will give an overview of all the APIs in use, how often they’re being used, and - for deprecated APIs - what release they’ll be removed in.
• Are cluster upgrades blocked if an API that is being removed is in use? Yes. Operators that are incompatible with future versions will also block updates.
• Is there a way to have the cluster automatically redirect or update the API call for the user? This depends on the API. OpenShift has several APIs like this, which will transparently mutate the request to be correct. However, not all have this capability.
• A viewer asked about backup strategies for OpenShift. We had a short conversation about this, including pointing out that we talked about this recently on the stream. The main takeaway is that etcd backups alone are not enough, but be sure to review the previous stream to take a more thorough look at backup for OpenShift.
• Rob does a live demo of the ApiRequestCounts CRD starting here. This is an excellent demo that walks through how to identify deprecated APIs in use and, very importantly, who or what is using them.
• Another viewer question: does there need to be special consideration for EUS releases with API deprecation and removal? No, you just need to be aware that going from EUS to EUS will result in all of the incremental changes happening in a condensed period of time. This means that there will probably be a larger number of changes wrapped into the update path.
• OLM will do some automatic conversion of validating and mutating webhooks.
• How can we stay informed of future API deprecations and removals? The cluster will alert you when you’re using deprecated APIs before they’re removed, but if you want to get even further ahead and know as soon as they’re planned, you’ll need to participate upstream.
• Did you know that blocked update edges are published using the errata process? If you’re subscribed to the OpenShift errata channel, you’ll get those notifications when they happen.