Automating OpenShift application DNS with external-dns and Red Hat’s Satellite 6 server

Automating OpenShift application DNS with external-dns and Red Hat’s Satellite 6 server

The configuration of DNS for applications in OpenShift Container Platform is largely handled by pre-configured wildcard zones that point to the routers in the OCP infrastructure. In the event that an application lands on an external IP for Ingress traffic or load balancer, DNS has to be configured for the endpoints. Endpoints are added or taken away as an application lifecycle changes. This creates some administrative overhead for DNS administration.

Red Hat’s Satellite 6 provides an all-in-one solution for managing your system’s lifecycle. This myriad of software includes the option of installing BIND for DNS services. This article will cover the use of BIND and ExternalDNS to automate the creation and removal of DNS records for external OCP services. ExternalDNS’s GitHub describes itself, “ExternalDNS synchronizes exposed Kubernetes Services and Ingresses with DNS providers.”

Bind uses RFC2136 for dynamic updates in DNS. Satellite 6 BIND has a pre-configured RNDC key for use with updating zone files.  The documentation page provides more information on how to configure RNDC for dynamic updates from start to finish.

BIND DNS Configuration

The file named.conf is the first configuration to examine for configuration of dynamic updates in DNS:

# cat /etc/named.conf

include "/etc/rndc.key";
controls {
 inet 10.x.y.2 port 953 allow { 10.x.y.151; } keys { "rndc-key"; };
 };
 options {
 include "/etc/named/options.conf";
 };
include "/etc/named.rfc1912.zones";
// Public view read by Server Admin
 include "/etc/named/zones.conf";
 

Open the files mentioned in the primary named.conf file.

The rndc key listed below will be the secret, name and algorithm provided to ExternalDNS:

# cat /etc/rndc.key

key "rndc-key" { (1)

algorithm hmac-md5; (2)

secret "T7wgNa7I1zL7oXSdd/4Pgg==";  (3)

};

The options file enables dnssec for updates:

# cat /etc/named/options.conf
..output omitted..
 dnssec-enable yes;
 dnssec-validation yes;
 ..output omitted..

Lastly, the permissions to allow updates with said key are allowed in the zones.conf file:

# cat /etc/named/zones.conf
..output omitted..
 zone "example.com" {
 type master;
 file "/var/named/dynamic/db.example.com";
 update-policy {
 grant rndc-key zonesub ANY;
 };
 };
 ..output omitted..
 

OpenShift Container Platform Configuration

Once the proper configuration of DNSSEC is enabled, the focus turns to ExternalDNS and setting up ingress traffic to external IP addresses on the OCP cluster in question. The OpenShift Container Platform documentation describes how to assign unique External IPs for Ingress traffic. Most of the cloud providers provide a facility to create Load Balancer services on their platform. This feature is only supported in a non-cloud deployment. The addresses assigned to the Ingress traffic needs to be configured in ipfailover.

The following command configures this:

# oc adm policy add-scc-to-user privileged -z ipfailover

# oc adm ipfailover --selector="beta.kubernetes.io/os=linux" \

--interface=ens192 --virtual-ips="10.x.y.249-253" \

--watch-port=80 --replicas=5 --create

On the master nodes in the cluster, the following options should be defined:

# cat /etc/origin/master/master-config.yaml
..output omitted..
 networkConfig:
 ingressIPNetworkCIDR: 10.x.y.248/29
 ..output omitted..
 

The AWS documentation for External-DNS provides a template for a POD deployment file:

# vi manifest.yml

apiVersion: v1

kind: ServiceAccount

metadata:

name: external-dns

---

apiVersion: rbac.authorization.k8s.io/v1beta1

kind: ClusterRole

metadata:

name: external-dns

rules:

- apiGroups: [""]

resources: ["services"]

verbs: ["get","watch","list"]

- apiGroups: [""]

resources: ["pods"]

verbs: ["get","watch","list"]

- apiGroups: ["extensions"]

resources: ["ingresses"]

verbs: ["get","watch","list"]

- apiGroups: [""]

resources: ["nodes"]

verbs: ["list"]

---

apiVersion: rbac.authorization.k8s.io/v1beta1

kind: ClusterRoleBinding

metadata:

name: external-dns-viewer

roleRef:

apiGroup: rbac.authorization.k8s.io

kind: ClusterRole

name: external-dns

subjects:

- kind: ServiceAccount

name: external-dns

namespace: default

---

apiVersion: extensions/v1beta1

kind: Deployment

metadata:

name: external-dns

spec:

strategy:

type: Recreate

template:

metadata:

labels:

app: external-dns

spec:

serviceAccountName: external-dns

containers:

- name: external-dns

image: registry.opensource.zalan.do/teapot/external-dns:latest

args:

- --source=service

- --provider=rfc2136

- --rfc2136-host=10.x.y.2

- --rfc2136-port=53

- --rfc2136-zone=example.com

- --rfc2136-tsig-secret='T7wgNa7I1zL7oXSdd/4Pgg=='

- --rfc2136-tsig-secret-alg=hmac-md5

- --rfc2136-tsig-keyname=rndc-key

- --rfc2136-tsig-axfr
# oc create -f manifest.yml
 

The relevant sections from the DNS configuration have been populated above. Next, create an example service that can be added to test DNS resolution and the record creation.

# example.yaml
apiVersion: v1
 kind: Service
 metadata:
 name: hello-openshift
 annotations:
 external-dns.alpha.kubernetes.io/hostname: hello-openshift.example.com
 spec:
 type: LoadBalancer
 loadBalancerIP: 10.x.y.252
 ports:
 - port: 8080
 name: http
 targetPort: 8080
 selector:
 app: hello-openshift
---
apiVersion: extensions/v1beta1
 kind: Deployment
 metadata:
 name: hello-openshift
 spec:
 template:
 metadata:
 labels:
 app: hello-openshift
 spec:
 containers:
 - image: docker.io/openshift/hello-openshift
 name: hello-openshift
 ports:
 - containerPort: 8080
 name: http
# oc create -f example.yml
 

Now, if the DNS server is monitored, the record is created and can be queried:

# watch dig @10.x.y.2 hello-openshift.example.com +short

10.x.y.252

# curl hello-openshift.example.com:8080

Hello OpenShift!

Lastly, clean up the environment and the record is removed from DNS:

# oc delete deployment.apps/hello-openshift

# oc delete service hello-openshift

# watch dig @10.x.y.2 hello-openshift.example.com +short

Alternatively, OCP external IP addresses can be used in this exercise as well. This will allow the use cloud providers and the administration of the external IP address is left to the user. On the master nodes:

On the master nodes in the cluster, the following options should be defined:

# cat /etc/origin/master/master-config.yaml
..output omitted..
 networkConfig:
 externalIPNetworkCIDRs: 10.x.y.248/29
 ..output omitted..
 

The service example will differ slightly:

apiVersion: v1

kind: Service

metadata:

name: hello-openshift

annotations:

external-dns.alpha.kubernetes.io/hostname: hello-openshift.example.com

spec:

externalIPs:

- 10.x.y.252

ports:

- port: 8080

name: http

targetPort: 8080

selector:

app: hello-openshift

This article has highlighted some functionality combining ExternalDNS and Red Hat’s Satellite 6 server. This functionality automates some typical DNS overhead in creating external service records for OpenShift administrators. In this example RFC2136, or dynamic updates in BIND server were utilized. ExternalDNS is an extensible solution that can cover many different DNS providers, see the documentation for more information about its uses.