Bonds in OpenShift Pods

Linux bonding enables higher bandwidth and high availability for networks. This is achieved by aggregating multiple network interfaces into a single logical interface. The bond-CNI brings interface bonding to OpenShift to be used inside pods. The primary use for pod bonding is to be used on top of SRIOV virtual functions.

Installing bond-cni

Bond-CNI is available in OpenShift 4.10 as tech-preview, and will be a supported feature starting from OpenShift 4.12. No further action is required to use it.

Configuring a bond network

A bond interface is defined using a NetworkAttachmentDefinition resource.

An example configuration is shown below

apiVersion: "k8s.cni.cncf.io/v1"
    kind: NetworkAttachmentDefinition
    metadata:
      name: bond-network
      namespace: bond-namespace
    spec:
      config: '{
      "type": "bond",
      "cniVersion": "0.3.1",
      "name": "bond-net1",
      "mode": "active-backup", 
      "failOverMac": 1,
      "linksInContainer": true,
      "miimon": "100",
      "mtu": 1500,
      "links": [
            {"name": "net1"}, {"name": "net2"}
        ],
      "ipam": {}
      }'

Below is an explanation of the individual attributes of the config section:

• type - cni type, must always be ‘bond’
• mode - the bonding mode, the following values are supported: “balance-rr”, “active-backup” ,”balance-xor”
• failOverMac - only valid for the “active-backup” mode, must be set to 1
• linksInContainer - indicates that the bond cni will use nterfaces inside the pod network namespace. The only supported value is “true”
• miimon - specifies the MII link monitoring frequency in milliseconds
• mtu - the mtu of the bond interface
• links - a list of network interfaces to be used. These interfaces must be present in the pod before the bond interface is created

Bonding aggregates multiple existing interfaces into a single logical bonded interface. Prior to creating the bond interface, the interfaces to be aggregated must be created. The bond interface will then be created using these interfaces.

Configuring a pod with a bond interface

A bond interface is configured as any other secondary interface, using the k8s.v1.cni.cncf.io/networks annotation. An example of a pod definition containing a bond network interface is shown below:

apiVersion: v1

    kind: Pod

    metadata:

      name: example-pod

      namespace: default

      annotations:

        k8s.v1.cni.cncf.io/networks: sriov/sriov-network, sriov/sriov-network, bond-namespace/bond-network,     

    spec:

       …

Note that the “k8s.v1.cni.cncf.io/networks” annotation contains a list of network attachment definitions. The bond interface is defined by the “bond-namespace/bond-network” element.

Bonding aggregates multiple existing interfaces into a single logical bonded interface. Prior to creating the bond interface, the interfaces to be aggregated must be created. The bond interface will then be created using these interfaces. To ensure that the aggregated interfaces are created before the bond interface, they must be defined in the list in the “links” attribute before the bond network attachment definition. Note that in the example above there are two elements in the list defined before the bond: 

annotations:

        k8s.v1.cni.cncf.io/networks: sriov/sriov-network, sriov/sriov-network, bond-namespace/bond-network

There are two elements defining the “sriov/sriov-network” secondary network. OpenShift will create the interfaces one by one, in the order defined in the annotation. The result of this will be that 2 interfaces defined by the “sriov-network” network attachment definition will be created before the bond interface is processed.

If not configured otherwise, the interface names in a pod are assigned automatically as “net<n>”, with n starting with 1. In the case of the pod example above the names of the interfaces would be “net1” and “net2”, with the bond interface being named “net3”. This is reflected in the bond network attachment definition “links” attribute, which specifies the “net1” and ”net2” interfaces: 

"links": [{"name": "net1"}, {"name": "net2"}]

The interfaces can be configured to use a specific name by adding a “@name” suffix to the pod’s “k8s.v1.cni.cncf.io/networks” annotation in the following way:

annotations:

        k8s.v1.cni.cncf.io/networks: sriov/sriov-network@ifc1, sriov/sriov-network@ifc2, bond-namespace/bond-network@bond

The “links” list should be modified as:

"links": [{"name": "ifc1"}, {"name": "ifc2"}]

and the bond being named “bond”.

Configuring SRIOV Interfaces

When SRIOV virtual functions are used in a bond in the “balance-rr” or “balance-xor” modes, the “trust” attribute for the virtual function must be set to “on” in the SriovNetwork.