Kubernetes NMState Operator Promoted to GA in OpenShift 4.10 on bare metal

Promotion to General Availability

The Kubernetes NMState Operator lets you report and update the network configuration of your cluster nodes. It uses a declarative configuration expressed in YAML, which is easy to follow and well documented.

The operator has been available as a Technology Preview since OpenShift 4.7. We have been working on making it ready for GA, ensuring it’s reliable and enterprise-ready, for us to confidently support it. We built a downstream CI with OpenShift, improved the code, fixed bugs and rolled out an updated documentation. This is the result of the amazing work that our developers, quality engineering team and tech writers have done.

What can you do with it?

The operator allows you to use NMstate for updating the nodes’ network configuration. There are plenty of configuration examples, review them here: https://nmstate.io/examples.html

Update your DNS

When we were gathering feedback we got many requests for what’s a very common and perhaps unsophisticated task: update the nodes’ DNS servers. Considering all the configurations you can make, this one looked straightforward, but we focused on making it work reliably in all cases anyway.

In our journey, we found that it now works great and that it has some areas to pay attention to:

• Always specify an interface when updating DNS resolvers
• Don’t use the br-ex interface since it’s used and owned by OVNKubernetes
• Use auto-dns: false to prevent your DHCP server from setting the resolvers

Here’s an working example:

apiVersion: nmstate.io/v1
kind: NodeNetworkConfiurationPolicy
metadata:
  name: enp0s3-dns-policy
spec:
  nodeSelector:
    kubernetes.io/hostname: worker-0
  desiredState:
    dns-resolver:
      config:
        search:
        - example.com
        server:
        - 8.8.8.8
    interfaces:
    - name: enp0s3
      type: ethernet
      state: up
      ipv4:
        auto-dns: false
        enabled: true
        dhcp: true

What about configuring nodes during the installation?

Another big feature introduced for the OpenShift bare metal installer is the ability to use NMState syntax to configure your nodes, one by one, at installation time.

This is a massive step forward in automation. Before this update, a DHCP server was always required when you wanted to fully automate your installation with the installer-provisioned infrastructure installation flow. Now you can specify your static IP assignments to your nodes with the same install-config.yaml file where you configure the rest of your bare metal cluster

Here’s an example where you can see the newly added section networkConfig in install-config.yaml that you can use for every node:

[...]
    hosts:
      - name: openshift-master-0
        networkConfig:
          routes:
            config:
            - destination: 0.0.0.0/0
              next-hop-address: 192.168.123.1
              next-hop-interface: enp0s4
          dns-resolver:
            config:
              server:
              - 192.168.123.1
          interfaces:
          - name: enp0s4
            type: ethernet
            state: up
            ipv4:
              address:
              - ip: 192.168.123.111
                prefix-length: 24
              enabled: true

Final notes

Many of our customers told us that the most important aspect of automated installations was not requiring a DHCP server, this is now possible thanks to this improvement. Another aspect many customers and users told us was the need to update the DNS servers in their cluster nodes, which is now also fully available. We now satisfy these and many others requirements with OpenShift 4.10.

This is just an introduction to what you can do starting in OpenShift 4.10 and, as you will see, Kubernetes NMState offers a wide range of additional host networking configurations.