In the last few years, several Red Hat customers have asked how to add a Web Application Firewall (WAF) to the OpenShift ingress to protect all externally facing applications.
A WAF is a Layer 7 capability that protects applications against some types of web-based attacks, including but not limited to Cross Site Request Forgery (CRSF), Cross-Site Scripting (XSS) and SQL injection (for a more comprehensive list of all known web based attacks, see here).Unfortunately, OpenShift does not have these capabilities included within the default ingress router, and as a result, alternate solutions must be pursued.
But now, with the advent of the Gateway API and a new ingress stack for OpenShift, things are changing. In this article, we will demonstrate how you can protect your applications with a WAF.
OpenShift, Gateway API and WASM
Note: As of this writing, the OpenShift Gateway API implementation is in a dev preview support status and the functionality may change in the future, including the underlying technologies.
OpenShift, as a product, will adopt the new Gateway API standard. This feature is in dev preview behind a feature gate and can be activated as described in this article.
For the purpose of this article, we do not need to analyze all of the new features of the Gateway API (a good summary for that can be found here). However, the feature that we are most concerned with is that the reverse proxy implementation of the Gateway API for OpenShift is Envoy. In fact, by enabling the Gateway API feature, a clusterwide OpenShift Service Mesh is currently installed automatically.
By having the power of the Istio at our disposal, we can easily inject new behavior in Envoy using WebAssembly (WASM) packaged plugins.WASM is a technology that compiles code such that it runs safely in a lightweight stack-based virtual machine.WASM was originally created to inject behavior in the browser to support sophisticated websites, but it turned out to be ideal for all kinds of plugin systems.To be compatible with Envoy, a WASM program must implement the proxy-wasm Application Binary Interface (ABI). This interface specification ensures that Envoy understands how to call the WASM program.
ModSecurity, OWASP WAF core rule set and Coraza
ModSecurity is a mature and stable Apache module that adds WAF capabilities to an Apache instance. ModSecurity has been adopted over time by other http servers and proxies products, such as ngnix via an appropriate connector.ModSecurity has recently been donated to Open Web Application Security Project (OWASP) and under this organization, will be maintained as an open source project with open and transparent governance.
One of the features of ModSecurity is that it can be configured to check for a set of rules via a Domain Specific Language (DSL) called SecLang. The OWASP organization maintains a default set of rules that cover the most common attack types. This set of rules is referred to as the OWASP core rule set.When configuring a WAF based on ModSecurity, it is a good practice to start with the OWASP core rule set and then tweak the ruleset based on one’s needs.
Coraza is a port of ModSecurity to the golang language (ModSecurity is originally written in C) with the objective of making ModSecurity more adopted by enterprises. Coraza Proxy WASM is an adaptation of Coraza to the proxy-wasm-ABI.
That being said, by leveraging these solutions, it should start to be clear how we are going to enhance OpenShift ingress with a WAF.
Putting it all together
The following diagram depicts the approach for injecting Coraza in the Envoy proxy which manages the ingress traffic:
In the diagram above, we can see in yellow, the CRs that we created, and in green, pods or groups of pods that are running. You can find this implementation in a declarative GitOps format within this repository.
At the bottom of the diagram, we can find the GatewayClass CR. When configured appropriately, it triggers the deployment of an ServiceMesh (Istio) control plane. Moving up higher in the diagram, we can see the Gateway CR. This manifest creates an Istio ingress gateway pod. When the Gateway CR is deployed the OpenShift Ingress Operator creates a wildcard DNS entry for this ingress endpoint. Additionally, we created a wildcard certificate using cert-manager to enable TLS for all the subdomains managed by this endpoint.
The last piece of configuration that the administrator needs to do is to inject the Coraza WAF code as a WASM plugin. This can be achieved with a WasmPlugin CR. Notice that this manifest references an OCI artifact containing the WASM code.
On the tenant side, we can deploy a simple application that demonstrates the use of a WAF now that the capability has been enabled. The example application can be found here. The key component in this application is the HttpRoute. By specifying the parentRefs section, we request the newly created Gateway to serve this route.
Now that we have enabled the Gateway API within OpenShift and configured the cluster to support our use case, and deployed the sample application, we can finally test and validate the functionality.
Let’s first get the current cluster base domain:
export cluster_base_domain=$(oc get ingress.config.openshift.io cluster --template={{.spec.domain}} | sed -e "s/^apps.//")
Then we can issue a request:
curl -I https://nginx-echo-headers.gwapi.${cluster_base_domain}
HTTP/2 200
server: istio-envoy
date: Fri, 10 May 2024 18:23:50 GMT
content-type: text/plain
x-envoy-upstream-service-time: 2
Now, let’s make a request that looks a little more suspicious:
curl -I "https://nginx-echo-headers.gwapi.${cluster_base_domain}/?arg=<script>alert(0)</script>"
HTTP/2 403
date: Fri, 10 May 2024 18:25:21 GMT
server: istio-envoy
The request was denied with a 403 error.
Within the Istio ingress gateway pod log, messages similar to the following will be displayed:
024-05-10T18:25:21.318531Z critical envoy wasm wasm log openshift-ingress.coraza-waf: [client "100.64.0.5"] Coraza: Warning. XSS Attack Detected via libinjection [file "@owasp_crs/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "7663"] [id "941100"] [rev ""] [msg "XSS Attack Detected via libinjection"] [data "Matched Data: XSS data found within ARGS_GET:arg: <script>alert(0)</script>"] [severity "critical"] [ver "OWASP_CRS/4.0.0-rc2"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-xss"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/152/242"] [hostname "10.129.2.11"] [uri "/?arg=<script>alert(0)</script>"] [unique_id "cmQXIPQegrompxpYJnD"]
126
…
Load test
By now, we know that the OWASP core rule set is being applied by our WAF./ But, was there a performance hit against the application? Let’s try to measure the delta with and without Coraza enabled.
To accomplish this task, we need to deploy another gateway without the plugin.
To generate load, we are going to use cassowary, a cross platform load testing tool. Here is our baseline, without Coraza:
podman run rogerw/cassowary:v0.14.1 -u https://nginx-echo-headers.gwapi-no-coraza.${cluster_base_domain} -c 10 -n 1500 -d 3
0 --disable-keep-alive
Starting Load Test with 1500 requests using 10 concurrent users
99% |███████████████████████████████████████ | [29s:0s] 30.056327229s
TCP Connect.....................: Avg/mean=17.74ms Median=17.00ms p(95)=22.00ms
Server Processing...............: Avg/mean=19.43ms Median=19.00ms p(95)=25.00ms
Content Transfer................: Avg/mean=0.00ms Median=0.00ms p(95)=0.00ms
Summary:
Total Req.......................: 1500
Failed Req......................: 0
DNS Lookup......................: 1.00ms
Req/s...........................: 49.91
Now, let’s see how the same load performs with Coraza enabled:
podman run rogerw/cassowary:v0.14.1 -u https://nginx-echo-headers.gwapi.${cluster_base_domain} -c 10 -n 1500 -d 30 --disable-keep-alive
Starting Load Test with 1500 requests using 10 concurrent users
99% |███████████████████████████████████████ | [29s:0s] 30.075051016s
TCP Connect.....................: Avg/mean=18.14ms Median=18.00ms p(95)=23.00ms
Server Processing...............: Avg/mean=33.46ms Median=33.00ms p(95)=40.00msEnvoy
Content Transfer................: Avg/mean=0.00ms Median=0.00ms p(95)=0.00ms
Summary:
Total Req.......................: 1500
Failed Req......................: 0
DNS Lookup......................: 1.00ms
Req/s...........................: 49.88
We can see that for this simple use case, the latency introduced by the WAF was around 14ms (33ms-19ms).
Also, we noticed that on the resource consumption side, the ingress gateway pod with Coraza (top in the picture below) consumes approximately one order of magnitude (x10) more memory and CPU.
Conclusions
In this article, we demonstrated an approach to add a WAF to our OpenShift ingress stack using the new Gateway API and injection via a WASM plugin. We used Coraza as the WAF implementation and the OWASP core rule set as our WAF policies. When attempting to deploy these assets in your own environment, please keep in mind that the Gateway API is still in dev preview in OpenShift and that the Coraza WASM plugin is not currently supported by Red Hat.
In addition to WAF and the core rule set, OWASP recommends sanitizing the HTTP headers following the OWASP Secure Headers Project “best practices”. These best practices can be tested at develop time, but one can also decide to sanitize the headers on the fly at run time. We built a custom WASM plugin to demonstrate this use case. The plugin can be found in this repository. This sample plugin attempts to satisfy most recommendations from the published HTTP Security Response Headers Cheat Sheet.
In our opinion, WASM is a promising approach for enhancing the behavior of the Envoy reverse proxy especially when Envoy is controlled by Istio (multiple plugins can be attached to the same Envoy instance). We also recommend investigating project Kuadrant in detail which uses WASM to add authz/authn and rate-limiting capabilities (and more) to an Envoy proxy.
À propos des auteurs
Trevor Box has been a Red Hatter since 2018 and concentrates on container platforms and their adoption.
Raffaele is a full-stack enterprise architect with 20+ years of experience. Raffaele started his career in Italy as a Java Architect then gradually moved to Integration Architect and then Enterprise Architect. Later he moved to the United States to eventually become an OpenShift Architect for Red Hat consulting services, acquiring, in the process, knowledge of the infrastructure side of IT.
Currently Raffaele covers a consulting position of cross-portfolio application architect with a focus on OpenShift. Most of his career Raffaele worked with large financial institutions allowing him to acquire an understanding of enterprise processes and security and compliance requirements of large enterprise customers.
Raffaele has become part of the CNCF TAG Storage and contributed to the Cloud Native Disaster Recovery whitepaper.
Recently Raffaele has been focusing on how to improve the developer experience by implementing internal development platforms (IDP).
Contenu similaire
Parcourir par canal
Automatisation
Les dernières nouveautés en matière d'automatisation informatique pour les technologies, les équipes et les environnements
Intelligence artificielle
Actualité sur les plateformes qui permettent aux clients d'exécuter des charges de travail d'IA sur tout type d'environnement
Cloud hybride ouvert
Découvrez comment créer un avenir flexible grâce au cloud hybride
Sécurité
Les dernières actualités sur la façon dont nous réduisons les risques dans tous les environnements et technologies
Edge computing
Actualité sur les plateformes qui simplifient les opérations en périphérie
Infrastructure
Les dernières nouveautés sur la plateforme Linux d'entreprise leader au monde
Applications
À l’intérieur de nos solutions aux défis d’application les plus difficiles
Programmes originaux
Histoires passionnantes de créateurs et de leaders de technologies d'entreprise
Produits
- Red Hat Enterprise Linux
- Red Hat OpenShift
- Red Hat Ansible Automation Platform
- Services cloud
- Voir tous les produits
Outils
- Formation et certification
- Mon compte
- Assistance client
- Ressources développeurs
- Rechercher un partenaire
- Red Hat Ecosystem Catalog
- Calculateur de valeur Red Hat
- Documentation
Essayer, acheter et vendre
Communication
- Contacter le service commercial
- Contactez notre service clientèle
- Contacter le service de formation
- Réseaux sociaux
À propos de Red Hat
Premier éditeur mondial de solutions Open Source pour les entreprises, nous fournissons des technologies Linux, cloud, de conteneurs et Kubernetes. Nous proposons des solutions stables qui aident les entreprises à jongler avec les divers environnements et plateformes, du cœur du datacenter à la périphérie du réseau.
Sélectionner une langue
Red Hat legal and privacy links
- À propos de Red Hat
- Carrières
- Événements
- Bureaux
- Contacter Red Hat
- Lire le blog Red Hat
- Diversité, équité et inclusion
- Cool Stuff Store
- Red Hat Summit