This post shows an example of using the Operator Lifecycle Manager (OLM) bundle deployment architecture to deploy an operator. The post demonstrates how various components of the operator-framework work together to deploy an operator.
A bundle is an operator packaging construct which contains an operator definition and manifests which ultimately determine how the operator is deployed onto a Kubernetes cluster. Bundles are the preferred mechanism within OLM going forward to package and deploy Operators.
Recently a migration from the original OLM package manifest format to the bundle format has begun. Operator developers are encouraged to develop their operators using the new bundle format.
For this exercise, I tested the OLM examples on both an Red Hat Openshift 4.6 cluster as well as a local kind 1.18 cluster. operator-sdk version 1.0 and OLM version 0.15.0 were used for this example.
Scaffolding the Operator
To start with, you can create an Operator using the operator-sdk to scaffold the starting point for an Operator implementation. In this example, the sample Operator being deployed was created with operator-sdk version 1.0.0. This particular version of operator-sdk has a capability to generate an Operator bundle. The operator-sdk commands used to scaffold the sample Operator include the following:
operator-sdk init --domain=example.com --repo=github.com/example-inc/doo-operator operator-sdk create api --group cache --version v1 --kind Doo --resource=true --controller=true
When operator-sdk generates or scaffolds out an operator, the Operator lacks logic about your application. It is up to the end user to include specifics for how to install and manage their app. Like any other operator, the resulting code is built into a container image. In this example, the operator image is named:
quay.io/username/doo-operator:v0.0.1
The Operator project structure created by the operator-sdk init command, will also create a Makefile that is then used throughout this post to perform various build and deployment tasks of the Operator.
This Operator image is built using the Makefile as shown in this example:
make docker-build docker-push IMG=quay.io/username/doo-operator:v0.0.1
Complete details on operator-sdk golang Operator scaffolding is found in this guide to building a Golang based Operator using Operator SDK.
Building the Operator Bundle Image
Apart from the Operator image itself, an Operator bundle is an OLM prescribed format for holding metadata about an operator. The metadata contains everything that Kubernetes needs to know in order to use the operator – its custom resource definitions (CRDs), Role-based access control (RBAC) roles and bindings required, dependency tree and other info. The operator-sdk generated Makefile enables you to create a bundle for your operator as follows:
make bundle IMG=quay.io/username/doo-operator:v0.0.1
This command generates an on-disk set of bundle manifests. Here is an example bundle directory structure that is generated:
bundle ├── manifests │ ├── cache.example.com_dooes.yaml │ ├── doo.clusterserviceversion.yaml │ └── doo-metrics-reader_rbac.authorization.k8s.io_v1beta1_clusterrole.yaml ├── metadata │ └── annotations.yaml └── tests └── scorecard └── config.yaml
The make bundle
command also creates a Dockerfile (bundle.Dockerfile
) which is used to build a "bundle image." The bundle image is an OCI image that essentially holds the generated on-disk bundle manifest and metadata files.
In this example, we are naming the Operator bundle image as follows:
quay.io/username/doo-operator-bundle:v0.0.1
To create and push the Operator bundle image, run these Makefile targets:
make bundle-build BUNDLE_IMG=quay.io/username/doo-operator-bundle:v0.0.1 make docker-push IMG=quay.io/username/doo-operator-bundle:v0.0.1
Building the Operator Index Image
Another OLM concept is the "index image," which is like a catalog of Operators. This container image acts to serve an application programming interface (API) which describes information about your sample operator and others in the catalog. The index image includes information from your bundle image by running an operator-registry command, opm, as follows:
opm index add --bundles quay.io/username/doo-operator-bundle:v0.0.1 --tag quay.io/username/doo-operator-index:v0.0.1 podman push quay.io/username/doo-operator-index:v0.0.1
The index image holds a sqlite database with bundle definitions, it also runs a grpc service when the image is executed. The grpc service lets consumers query the sqlite database for information about the operators this index contains.
You can download the opm command from the operator-registry repository found on the GitHub operator-framework/operator-registry repository.
Running the Bundle Index Image
At this point, I am assuming you have OLM installed on your Kubernetes cluster. OLM is installed by default on Openshift clusters. Refer to the operator-sdk’s olm install
command to manually install OLM on a non-Openshift cluster.
Once you have an index image, you deploy it by creating an OLM CatalogSource resource. For our sample operator, we create the following CatalogSource as follows:
cat <<EOF | kubectl create -f - kind: CatalogSource metadata: name: doo-operator namespace: operators spec: sourceType: grpc image: quay.io/username/doo-operator-index:v0.0.1 EOF
This CatalogSource is created in an existing namespace, created by OLM, named operators
; on OpenShift clusters, this namespace is called openshift-operators
. When you create the CatalogSource, it causes the index image to be executed as a Pod. You can view it as follows:
kubectl -n operators get pod --selector=olm.catalogSource=doo-operator NAME READY STATUS RESTARTS AGE doo-operator-79x8z 1/1 Running 0 136m
You can look at the Pod log to make sure the image is serving the grpc API:
kubectl -n operators logs pod/doo-operator-79x8z time="2020-10-05T13:17:04Z" level=info msg="Keeping server open for infinite seconds" database=/database/index.db port=50051 time="2020-10-05T13:17:04Z" level=info msg="serving registry" database=/database/index.db port=50051
Examining the Index Image
Once the index image is running, you can query it using a tool like grpcurl
. For example, you can run these commands to directly access the grpc API:
kubectl -n operators port-forward pod/doo-operator-79x8z 50051:50051
In another terminal, run:
grpcurl -plaintext localhost:50051 api.Registry/ListPackages { "name": "doo" }
An alternative to the grpcurl command is to query the index image’s embedded sqlite database file. Notice that the database file is stored within the index image itself and is immutable. You can extract the sqlite database file used by the image using the following command:
kubectl -n operators cp doo-operator-79x8z:/database/index.db .
You can then use the sqlitebrowser UI to view the contents of the index database:
sqlitebrowser index.db
Using grpcurl or sqlitebrowser to view the index image’s contents is a means to debug what the operator description and manifests include and verify what version of an operator will be deployed when a Subscription is created.
Deploying
You use an OLM Subscription resource to trigger a specific operator deployment by OLM. The following command creates a Subscription which triggers the deployment of our sample operator:
cat <<EOF | kubectl create -f - apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: doo-subscription namespace: operators spec: channel: alpha name: doo source: doo-operator sourceNamespace: operators EOF
Notice that the Subscription is created in the already existing operators
namespace. This will cause OLM to create the sample operator into the operators
namespace. Note, on Openshift clusters, the operators
namespace is named openshift-operators
. Keep in mind that a Subscription needs to be in the same namespace as your CatalogSource.
Verify
You can verify that the sample Operator is running using the following commands:
kubectl -n operators get subscription NAME PACKAGE SOURCE CHANNEL doo-subscription doo doo-operator alpha kubectl -n operators get csv NAME DISPLAY VERSION REPLACES PHASE doo.v0.0.1 doo-operator 0.0.1 Succeeded kubectl -n operators get pod NAME READY STATUS RESTARTS AGE doo-controller-manager-6c4bdf7db6-jcvpn 2/2 Running 0 10m
Test
You can test the sample operator by creating a CustomResource that the sample operator is watching. Here is an example of a CustomResource for the sample operator:
You can create the CustomResource in the default
namespace as follows:
cat <<EOF | kubectl -n default create -f - { "apiVersion": "cache.example.com/v1", "kind": "Doo", "metadata": { "name": "doo-sample" }, "spec": { "foo": "bar" } } EOF
You should see that your sample operator, running in operators
namespace, has responded to this CustomResource being created in default
namespace by looking at the sample operator log:
kubectl -n operators logs pod/doo-controller-manager-6c4bdf7db6-jcvpn -c manager 2020-10-05T13:29:52.175Z DEBUG controller Successfully Reconciled{"reconcilerGroup": "cache.example.com", "reconcilerKind": "Doo", "controller": "doo", "name": "doo-sample", "namespace": "default"}
OperatorGroup
The examples shown in this blog utilize namespaces created by OLM when you install OLM. You might instead wish to isolate your operator deployments into a namespace that you create instead. For this, before you create your CatalogSource and Subscription, you will need to create a unique namespace (e.g. jeff-operators
) and OperatorGroup (e.g. jeff-operators
):
kubectl create namespace jeff-operators cat <<EOF | kubectl create -f - apiVersion: operators.coreos.com/v1 kind: OperatorGroup metadata: name: jeff-operators namespace: jeff-operators status: lastUpdated: "2020-10-07T13:44:54Z" namespaces: - "" EOF
Details on how OperatorGroups work are found here, note that a CatalogSource and Subscription need to be created in a namespace that contains an OperatorGroup.
Future SDK OLM Bundle Automation
In upcoming operator-sdk releases, there is planned a run bundle
command that performs many of the steps above using an temporary bundle index image. This new command should be useful for developers needing to test their operators using the OLM bundle architecture. Here are some examples of that new command:
operator-sdk run bundle quay.io/username/doo-operator-bundle:v0.0.1
Conclusion
The OLM bundle architecture provides an advanced mechanism to describe, publish, and deploy operators on k8s clusters.
Acknowledgements
Thank you to Eric Stroczynski and Jesus Rodriguez for reviewing this blog post.
Sobre el autor
Jeff McCormick works in the Openshift Operator Framework group at Red Hat, focused on Operator technologies.
Navegar por canal
Automatización
Las últimas novedades en la automatización de la TI para los equipos, la tecnología y los entornos
Inteligencia artificial
Descubra las actualizaciones en las plataformas que permiten a los clientes ejecutar cargas de trabajo de inteligecia artificial en cualquier lugar
Nube híbrida abierta
Vea como construimos un futuro flexible con la nube híbrida
Seguridad
Vea las últimas novedades sobre cómo reducimos los riesgos en entornos y tecnologías
Edge computing
Conozca las actualizaciones en las plataformas que simplifican las operaciones en el edge
Infraestructura
Vea las últimas novedades sobre la plataforma Linux empresarial líder en el mundo
Aplicaciones
Conozca nuestras soluciones para abordar los desafíos más complejos de las aplicaciones
Programas originales
Vea historias divertidas de creadores y líderes en tecnología empresarial
Productos
- Red Hat Enterprise Linux
- Red Hat OpenShift
- Red Hat Ansible Automation Platform
- Servicios de nube
- Ver todos los productos
Herramientas
- Training y Certificación
- Mi cuenta
- Soporte al cliente
- Recursos para desarrolladores
- Busque un partner
- Red Hat Ecosystem Catalog
- Calculador de valor Red Hat
- Documentación
Realice pruebas, compras y ventas
Comunicarse
- Comuníquese con la oficina de ventas
- Comuníquese con el servicio al cliente
- Comuníquese con Red Hat Training
- Redes sociales
Acerca de Red Hat
Somos el proveedor líder a nivel mundial de soluciones empresariales de código abierto, incluyendo Linux, cloud, contenedores y Kubernetes. Ofrecemos soluciones reforzadas, las cuales permiten que las empresas trabajen en distintas plataformas y entornos con facilidad, desde el centro de datos principal hasta el extremo de la red.
Seleccionar idioma
Red Hat legal and privacy links
- Acerca de Red Hat
- Oportunidades de empleo
- Eventos
- Sedes
- Póngase en contacto con Red Hat
- Blog de Red Hat
- Diversidad, igualdad e inclusión
- Cool Stuff Store
- Red Hat Summit