What is etcd?

etcd (pronounced et-see-dee) is an open source, distributed, consistent key-value store for shared configuration, service discovery, and scheduler coordination of distributed systems or clusters of machines. etcd helps to facilitate safer automatic updates, coordinates work being scheduled to hosts, and assists in the set up of overlay networking for containers.

etcd is a core component of many other projects. Most notably it is the primary datastore of Kubernetes, the de facto standard system for container orchestration. By using etcd, cloud-native applications can maintain more consistent uptime and remain working, even in the face of individual servers failing. Applications read data from and write data to etcd; it distributes configuration data providing redundancy and resiliency for the configuration of nodes.

As the primary datastore of Kubernetes, etcd stores and replicates all Kubernetes cluster states. Since it is a critical component of a Kubernetes cluster it is important that etcd has a reliable approach to its configuration and management.

Because etcd is a distributed consensus-based system, the cluster configuration of etcd can be complicated. Bootstrapping, maintaining quorum, reconfiguring cluster membership, creating backups, handling disaster recovery, and monitoring critical events are complex and tedious tasks requiring specialized expertise.

All of this is made easier through use of the etcd Operator.

Using an Operator, which represents human operational knowledge, makes etcd easier to use on Kubernetes or a Kubernetes container platform, like Red Hat OpenShift. The etcd Operator manages etcd within the Operator Framework and serves to simplify etcd cluster configuration and management.

The etcd Operator installs with a single command and allows users to configure and manage the complexities of etcd using simple declarative configuration that creates, configures, and manages etcd clusters.

The etcd Operator provides the following features:

• Create/Destroy - Instead of specifying tedious configuration settings for each etcd member, users only need to specify the size of the cluster.
• Resize - Users only need to modify the size in spec and the etcd Operator will take care of deploying, destroying and/or reconfiguring cluster members.
• Backup - The etcd Operator performs backups automatically and transparently. Users only need to specify the backup policy. For example: Backup every 30 minutes and keep the last 3 backups.
• Upgrade - Upgrading etcd without downtime is a critical but difficult task. Doing it with the etcd Operator simplifies operations and avoids common upgrade errors.

The etcd Operator simulates human operator behaviors in 3 steps: observe, analyze, and act.

1. The Operator observes the current cluster state by using the Kubernetes API.
2. Then it finds the differences between the desired state and current state.
3. Finally, it fixes the difference through one or both of the etcd cluster management APIs or the Kubernetes API.

etcd was created by the CoreOS team in 2013, and has been maintained by Red Hat engineers working alongside a team of peers from across the industry. 

In 2018 Red Hat and the CoreOS team gave the etcd community project to the Cloud Native Computing Foundation (CNCF), a vendor-neutral organization housed under The Linux Foundation to drive the adoption of cloud-native systems. Given that etcd powers every Kubernetes cluster, this move brings etcd to the community that relies on it most at the CNCF.

Red Hat plans to continue participating in developing etcd, especially as a part of our enterprise Kubernetes product, Red Hat OpenShift. Red Hat was also one of the early supporting companies that helped to launch CNCF in 2015 and has since worked alongside the foundation and community on the development and growth of cloud-native technologies. With our history and dedication to open source software and community-driven development, etcd can benefit the community even further within CNCF.