What is KubeLinter?

What is KubeLinter?

The KubeLinter is an open-source command-line interface to identify misconfigurations in Kubernetes objects. KubeLinter offers the ability to integrate checks on Kubernetes YAML files and Helm charts before deployment into a Kubernetes cluster. With 19 standard built-in checks and the room to configure your own, you get immediate feedback about misconfigurations and Kubernetes security violations.

How Do I Use It?

KubeLinter requires very minimal configuration and is used by executing commands in your command-line shell. Developed using Go, KubeLinter is comparable to kubectl and made with a few of the same packages.

Microsoft Windows

• Command Prompt
• PowerShell

Unix-like systems

• Bash
• Zsh
• Tcsh
• And more 

How Do I Install KubeLinter?

Using Brew

If you have Homebrew for macOS or LinuxBrew, you’re in luck. KubeLinter installs with a simple command:

brew install kube-linter

Using the Latest Binary

You can download the latest binary from Releases and add it to your PATH. These binaries allow easy use in container images and CI pipelines.

Using Go

If you prefer using Go, you can use KubeLinter after the following command:

GO111MODULE=on go get golang.stackrox.io/kube-linter/cmd/kube-linter

Building From Source

Lastly, you can always build from source:

1. Clone the KubeLinter repository.
2. Compile the source code using the makefile.
3. Move the binary (located in the .gobin folder).
4. Lastly, verify your version to ensure you’ve successfully installed KubeLinter.

.gobin/kube-linter version

Or,

kube-linter version

What Can I Do with KubeLinter

Now that the formalities are out of the way, it’s time to get started.

Type kube-linter into the command line, and you will see an output that looks remarkably similar to executing kubectl.

Usage: kube-linter [command]

Available Commands: checks View more information on lint checks help Help about any command lint Lint Kubernetes YAML files and Helm charts templates View more information on check templates version Print version and exit

Flags: -h, --help help for kube-linter

Use "kube-linter [command] --help" for more information about a command.

KubeLinter has five commands:

1. checks 2. help 3. lint 4. templates 5. version

The second and fifth commands are relatively straight forward. The help command gives further context, and the version command will provide the current version of the CLI. The kube-linter check list command will list all of the default policies that come with KubeLinter, and templates will show what you can do with them using custom checks.

Now, don’t jump ahead to custom checks yet. Let’s start by focusing on the core KubeLinter commands.

Linting Your YAMLs

The lint command requires at least one argument and comes with additional flags. To lint a single file, provide KubeLinter to your Kubernetes yaml file:

kube-linter lint path/to/yaml-file.yaml

Or you can lint an entire directory and its sub contents:

kube-linter lint path/to/directory/containing/yaml-files/

Lastly, for a Helm chart:

kube-linter lint path/to/directory/containing/Chart.yaml-file/

By allowing for a single file, folder, and Helm charts, KubeLinter gives flexibility in how its policies can be used and applied. Extremely useful considering there are many ways that YAML files and Helm charts are stored.

In the kube-linter-walkthrough GitHub repository, the sock-shop application configuration files are waiting to be checked. From here on out, all examples will be from the walkthrough repository.

Applying Policies

To learn about the 19 default checks, go to the KubeLinter documentation or run thekube-linter check list command for the complete list. Either way, you can review the 19 checks available to you, however, not all 19 checks are a part of the default lint command.

As you can see, there are 13 policies enabled by default and six that are not. Viswajith Venugopal commented about the decision around the choice of 13 default policies:

The idea was that it should be effortless to use out-of-the-box without a config file — I didn’t want people running it and feeling overwhelmed with the noise. They could use kube-linter lint --add-all-built-in to get this behavior (and then rely on the config file or explicit --exclude args) to remove what they don’t want.

Since KubeLinter applies the 13 checks by default, you will require an extra flag to your argument for more options. There are two main flags to add to the kube-linter lint <filepath> command. The –add-all-built-in flag will enable all of the 19 checks, and the --do-not-auto-add-defaults flag will disable all of the checks. You can put these flags to work using the example repo:

Input:

kube-linter lint manifests/compliance-yamls/non-compliant-app.yaml

Output:

Error: found 8 lint errors

And checking the same YAML but using the --add-all-built-in flag:

Input:

kube-linter lint manifests/compliance-yamls/non-compliant-app.yaml --add-all-built-in

Output:

Error: found 12 lint errors

If you run the command in your terminal, you would see the error output and a detailed explanation of each error. The output will look like the following:

manifests/compliance-yamls/non-compliant-app.yaml: (object: my-namespace/non-compliant apps/v1, Kind=Deployment) container "nginx" has memory limit 0 (check: unset-memory-requirements, remediation: Set your container's memory requests and limits depending on its requirements. Seehttps://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limitsfor more details.)

Each error will detail the object, the error that it has flagged, and the remediation steps. Part of KubeLinter’s goal is to educate users on the path forward for specific issues. Since many security tools expect security knowledge out the gate, the developers felt the giving guidance and extra reading would help educate the beginner/developer who may not have as much Kubernetes expertise.

And of course, you can just as quickly disable them using the --do-not-auto-add-defaults flag. For example:

Input:

kube-linter lint manifests/compliance-yamls/non-compliant-app.yaml --do-not-auto-add-defaults

Output:

Warning: no checks enabled.

I’m sure you’re thinking, “Why would I disable all policies? This defeats the purpose, doesn’t it?” and you would be right! That is where the --include and --exclude flags come in.

Of all the checks to run, the non-compliant-app violates ten of them (12 errors since CPU and memory have a limit and request):

1. env-var-secret
2. no-liveness-probe
3. no-read-only-root-fs
4. no-readiness-probe
5. non-existent-service-account
6. required-annotation-email
7. required-label-owner
8. run-as-non-root
9. unset-cpu-requirements
10. unset-memory-requirements

However, for now, you only care about the run-as-non-root check. In that case, you can disable all of the other checks and include only the checks you care about.

Input:

kube-linter lint manifests/compliance-yamls/non-compliant-app.yaml --do-not-auto-add-defaults --include run-as-non-root

Output:

Error: found 1 lint errors

Of course, you can always enable all checks and disable specific ones as well. For a single developer, using the CLI in this way is useful. However, CLI commands are imperative and can change between developers. The issues compound as teams grow and admins manage policies across teams. To enable policy adoptions at scale, KubeLinter needs to be repeatable, flexible, and portable, where the config file comes in.

Configuring Policies

One of the seven flags available for the kube-linter lint command is the --config flag.

KubeLinter can use a configuration YAML file to manage checks declaratively. Let’s start with a couple of basic config files. In the kube-linter-walkthrough repository, there are a few config files that highlight the flexibility of KubeLinter.

The config_addAllBuiltIn.yaml is setting all the built-in checks set to true. And vice-versa, the config_doNotAutoAddDefaults.yaml removes all of the checks. If you point the CLI command to these files, you will get the same result using the previous section’s flags.

Input:

kube-linter lint manifests/compliance-yamls/non-compliant-app.yaml --config configs/config_addAllBuiltIn.yaml

Output:

Error: found 12 lint errors

Input:

kube-linter lint manifests/compliance-yamls/non-compliant-app.yaml --config configs/config_doNotAutoAddDefaults.yaml

Output:

Error: found 0 lint errors

And just like the --include and --exclude flags, you can set the policies you wish to include or exclude. With the addAllBuiltIn YAML, you can exclude any check and do the opposite in the “do not add all config”, allowing you to use any mix of the 19 policies available.

The config file allows flexibility and repeatability in how policies are applied. Config files can be used in different pipeline stages, enabling new user awareness of the default policies before enforcement before production. With that in mind, the real flexibility of KubeLinter is in the ability to create custom checks and ignore specific cases.

Using Kubernetes Annotations for Ignoring Specific Cases

As you move to create policy enforcement across teams and users, there will be specific use cases that will require an exception. Exceptions need documentation, and what better way to use the declarative nature of Kubernetes than to make your code the documentation.

Kubernetes annotations allow for the injection of metadata into Kubernetes objects that can be used by external tools but not used by the Kubernetes API server. The application of annotations varies, from simple use-cases such as noting contact information to more complicated cases like side-car auto-injection in a namespace.

The core use of annotations is to allow users to give more context to their Kubernetes Objects. Using annotations to declare the specific exceptions, you can promote application knowledge communication throughout your pipeline since the developers or DevOps engineers know best about what each container requires.

For example, in the case below, the cart database requires root access on the host. Usually, this would be flagged by KubeLinter. However, for this demonstration, the database will be allowed to write to the host. For this example, the command points to a config_db.yaml that only has the run-as-non-root check enabled.

kube-linter lint manifests/carts-db/carts-db-dep.yaml --config configs/config_db.yaml … Error: found 1 lint errors

Now, it’s time to remove the last lint error. To ensure KubeLinter ignores any checks, you need to add a specific annotation.

ignore-check.kube-linter.io/<check-name>

And since you want to ignore the run-as-non-root check, simply add that check name to the annotation.

ignore-check.kube-linter.io/run-as-non-root: ""

Lastly, this annotation allows for the addition of a description to be associated with the key. It is highly recommended that there is a detailed description if any check is ignored.

ignore-check.kube-linter.io/run-as-non-root: "this deployment needs privileged access to the host"

After adding the annotation, you can rerun the command and watch the changes.

kube-linter lint manifests/carts-db/carts-db-dep.yaml --config configs/config_db.yaml … No lint errors found!

Customizing the Default Checks

Up to this point, you have worked with the 19 checks, enabled and disabled them, created config files to repeat the checks, and used annotations to ignore specific checks.

Another piece of KubeLinter is the personalization of the default checks. KubeLinter checks are built off templates, and you can adjust the parameters of these templates to suit your needs. Below are the templates available and the parameters that can be altered.

For example, the required-annotation check contains two alterable parameters, the key and value parameters. In the config_customChecks.yaml file, there are two custom checks as examples demonstrating this ability:

customChecks: - name: required-annotation-responsible template: required-annotation params: key: kube-linter/annotation remediation: please add the "kube-linter/annotation" annotation to the deployment - name: required-label-release template: required-label params: key: team remediation: please add a team label to the service scope: objectKinds: - Service

With particular templates, you can change the Kubernetes object you are applying the check to. For example, with the required-label-release check, you can change the scope to a service where it would default to a deployment. Putting this check into action, if you run:

kube-linter lint manifests/carts-db/carts-db-dep.yaml --config configs/config_customChecks.yaml

You will get the error message that matches your custom rule.

manifests/carts-db/carts-db-dep.yaml: (object: sock-shop/carts-db apps/v1, Kind=Deployment)no annotation matching "kube-linter/demo=<any>" found (check: required-annotation-responsible, remediation: please add the "kube-linter/demo" annotation to the deployment)

Error: found 1 lint errors

Lastly, the remediation value can be altered for your needs. It enables pointed feedback to developers or users of the configuration file. Useful to help users get accustomed to a team or organizational best practices.

Using KubeLinter in a Pipeline

KubeLinter can be integrated into a new or existing pipeline, and the Go binary provides a simple download, install, and execution process. The strength of KubeLinter in these pipelines is implementing the configuration files and CLI commands to customize the checks for different YAML files.

In the configs directory, there are two configurations to illustrate this. The config_carts.yaml is configured for the /manifests/carts/ yaml files, and the config_carts-db.yaml is for the /manifests/carts-db/ yaml files. Both configuration files have the same custom checks, however, the application of the default checks is different. This example illustrates how you can enforce team, project, or organization defaults while simultaneously customizing for a specific use case.

You can simulate the example GitHub Action by running simultaneous commands:

1. kube-linter lint manifests/carts-db/ --config configs/config_db_carts.yaml

2. kube-linter lint manifests/orders-db/ --config configs/config_db_orders.yaml

The previous command will output different errors due to the difference in config files. However, you can use the flexibility of these pipeline for greater control.

By default, most pipelines will fail if there is a non-zero exit code. So, if both commands are executed, the second command’s output will not be seen unless the first command passes. To get around this, break the two commands into separate steps and create multiple workflow stages.

For example, the demo repository contains two similar jobs in a single GitHub action. The first is the “test” job. This job will continue in the event of an error. The second job is a “staging” job that will stop in the event of a non-zero exit code. You could use this to apply the checks to the development branches without breaking pipelines, allowing developers to learn about violations without breaking their pipelines. The pipeline will then exit and return an error if those issues are not fixed before the staging job.

There are many different configurations available to you depending on the CI tool you are using. The main takeaway is the flexibility that KubeLinter gives you in applying these security checks.

What’s Next

As the community continues to grow, the KubeLinter development team wants to hear from you.

1. What new checks do you want to see?
2. Do you want to see templates applied to other objectKinds?
3. Do you want more example configuration files?

The team would love to hear from you! Make sure that you:

1. Star and watch the KubeLinter public repository.
2. Join KubeLinter on slack to get quicker feedback.
3. Open an issue or request and let your feedback be known.

If you are interested in other writeups, take a look at Steven Vaughan-Nichols write-up in The New Stack or Jessica Cherry’s thoughts at Opensource.

Thanks for reading, and stay safe.