Establish secure routes and TLS termination with wildcard certificates

In part one of this series, 3 ways to encrypt communications in protected environments with Red Hat OpenShift, I explored the basics of establishing secure routes and transport layer security (TLS) termination. I also briefly covered three approaches to establishing a secure end-to-end TLS pathway. This article will delve deeper into the first of these approaches, route reencryption with route-specific or wildcard certificates, and illustrate it with a practical example

[ Discover ways enterprise architects can map and implement modern IT strategy with a hybrid cloud strategy. ]

What are wildcard certificates?

An SSL/TLS wildcard certificate is a single public key certificate with a wildcard character (*) in the domain name field. This allows the certificate to be used with multiple subdomain names (hosts) of the same base domain. Wildcard certificates can be a more cost-effective and convenient option than conventional certificates, which are required for each subdomain. They are classified based on validation level and the number of domains and servers that can use them. Multidomain wildcard certificates further simplify the process and reduce costs by allowing administrators to secure multiple domains and their subdomains with a single certificate.

For example, a wildcard certificate for *.{navy}.mil could be used for www.navy.mil, mail.navy.mil, database.navy.mil, and any other subdomain name in the navy.mil domain.

By default, the Red Hat OpenShift Container Platform uses the Ingress Operator to create an internal certificate authority (CA) and issue a wildcard certificate valid for applications under the .apps subdomain. The web console and the command-line interface (CLI) use this certificate. You can replace the default wildcard certificate with one issued by a public CA included in the CA bundle provided by the container userspace. This approach allows external clients to connect to applications running under the .apps subdomain securely.

You can replace the default ingress certificate for all applications under the .apps subdomain. After replacing the certificate, all applications, including the web console and CLI, will be encrypted using the specified certificate.

[ Modernize your IT with managed cloud services. ]

One clear benefit of using a wildcard certificate is that it minimizes the effort of managing and securing multiple subdomains.

However, this convenience comes at the cost of sharing the same private key across all managed subdomains.

When an OpenShift cluster hosts the applications, a compromised key would potentially impact the security of all subdomains instead of just one. You can obtain an Extended Validation Multi-Domain certificate to address this issue. An EV Multi-Domain cert secures multiple domains under a single certificate while allowing it to secure as many subdomains as needed—thus delivering the most usage versatility.

Replace the default wildcard certificate

The OpenShift documentation for Replacing the default ingress certificate provides the following instructions.

Prerequisites

• Wildcard certificates for the fully qualified .apps subdomain and its corresponding private key. Each should be in a separate Privacy Enhanced Mail (PEM) format file.
• The private key must be unencrypted. If the key is encrypted, decrypt it before importing it into the OpenShift Container Platform.
• The certificate must include the subjectAltName extension showing *.apps.<clustername>.<domain>.
• The certificate file can contain one or more certificates in a chain. The wildcard certificate must be the first certificate in the file. Any intermediate certificates can follow it, and the file should end with the root CA certificate.
• Copy the root CA certificate into an additional PEM format file.

[ Hybrid cloud and Kubernetes: A guide to successful architecture ] 

Procedure

1. Create a config map that includes only the root CA certificate used to sign the wildcard certificate:

$ oc create configmap custom-ca \
     --from-file=ca-bundle.crt=</path/example-ca.crt> -n openshift-config

where </path/example-ca.crt> is the path to the root CA certificate file on the local filesystem.

2. Update the cluster-wide proxy configuration with the newly created config map:

$ oc patch proxy/cluster \
     --type=merge \
     --patch='{"spec":{"trustedCA":{"name":"custom-ca"}}}'

3. Create a secret that contains the wildcard certificate chain and key:

$ oc create secret tls <secret> --cert=</path/cert.crt> \
     --key=</path/cert.key> -n openshift-ingress

Where:

• <secret> is the secret's name containing the certificate chain and private key.
• </path/cert.crt> is the path to the certificate chain on the local filesystem.
• </path/cert.key> is the path to the private key associated with this certificate.

4. Update the Ingress Controller configuration with the newly created secret:

$ oc patch ingresscontroller.operator default --type=merge -p \
     '{"spec":{"defaultCertificate": {"name": "<secret>"}}}'  -n openshift-ingress-operator

Where:

• <secret> is the name of the secret that will contain the certificate chain and private key (same as in step 3).

While this guide discourages wildcard certificates and expresses preference toward certain architectures (for example, service mesh with mTLS or pass-through secure route with named certificates), I want to initiate a conversation to help organizations to find the right solution within their own technology landscape and requirements.

[ Learn more about cloud-native development in the eBook Kubernetes Patterns: Reusable elements for designing cloud-native applications. ]

Use named certificates and certificate management

A named TLS certificate is a certificate for a single domain. Single-domain SSL certificates are available at three validation levels:

• Domain Validation: The certificate authority verifies the ownership of a domain and ensures that the domain name belongs to the SSL applicant.
• Organization Validation: Organization validation establishes an organization's identity and requires business-related documents to verify the business' existence.
• Extended Validation (EV): EV follows a strict and lengthy process, including domain and organization verification. The process checks the entity's legal, operational, and physical existence by following EV guidelines that define vetting policies and principles.

Named certificates deliver the most control and flexibility over secure access to the domains and subdomains used by the applications hosted on the OpenShift platform. You can set the right validation level for the requested certificate and apply it to a specific domain or subdomain as required.

However, dealing with named certificates is usually a manual process requiring the operator(s) to run periodic checks to ensure the certificates are valid and up to date and renew them before expiration. Faced with a potentially large number of named certificates to manage, most organizations turn to tools for help with certificate management.

One recommended way to manage the certificates within the OpenShift cluster is to use the Cert-Manager tool deployed by the Operator. Cert-Manager adds certificates and certificate issuers as resource types in Kubernetes clusters, simplifying obtaining, renewing, and using those certificates. The OpenShift 4.x platform supports this via the Operator framework, making it possible to provide Certificates as a Service to developers working within the OpenShift cluster.

[ Related reading: How to configure your CA trust list in Linux ]

Cert-Manager is based on a plug-in architecture and can support multiple CA implementations. It currently supports the following types of CAs:

• Externally supplied CAs (either root CAs or sub-CAs)
• Systems that support the Automated Certificate Management Environment (ACME) protocol, such as Let's Encrypt
• Integration with Hashicorp Vault

By using Cert-Manager, you can automate the process of obtaining and renewing certificates from these CAs and use them to secure your applications and services.

The Cert-Manager operator for OpenShift is not installed in OpenShift Container Platform by default. It is usually deployed from the OperatorHub and requires the user to have cluster-admin privileges to install and operate using the Web Console.

The following is an example diagram (borrowed from an article by Raffaele Spazzoli) of a request for a certificate with two Subject Alternative Names (SANs)—one for external connections coming through a secure pass-through route, and another for internal connections through the service. The certificate will be issued by the Cert-Manager-supported certificate issuers previously configured by the administrator.

Use Cert-Manager

The following articles demonstrate how to create and manage TLS certificates with Cert-Manager on the OpenShift platform:

• How to create a TLS/SSL certificate with a Cert-Manager Operator on OpenShift
• Let's Encrypt certificate management on OpenShift using Cert-Manager Operator

The next article in this series explores how using Mutual Transport Layer Security (mTLS) together with OpenShift Service Mesh can provide an additional layer of security to the service mesh. This approach helps protect sensitive data, ensures that only authorized services can communicate with each other, and helps improve compliance with security policies. And the final article will examine using Keycloak SSO and TLS to build secure routes.