Introduction to certificate compression in GnuTLS

In a world where the Internet is the most common source of information and interconnection, it must be as efficient and secure as possible.

Ideally, the only information transmitted over a network would be the data requested by a user. However, this ideal is far from reality. Many devices must communicate and understand each other; hence protocols and administrative data are necessary.

[ Learn how to manage your Linux environment for success. ]

Transport Layer Security (TLS) is a widely supported cryptographic protocol that protects data privacy over the Internet. In TLS, certificate chains are one of the largest chunks of administrative data. These are usually several kilobytes large and require the exchange of multiple packets. Reducing the size of these chains would reduce the number of packets the protocol sends, improving the latency and overall performance of the initial TLS handshake. There have been attempts to do this by enabling protocol-level compression. However, this has proven to be insecure, as demonstrated by the CRIME attack.

The CRIME attack exploits information inherently leaked by the compression of protocol headers and secret data. If you were to compress secret strings foofoo and foobar, string foofoo would result in a shorter ciphertext than string foobar. An attacker could then observe these changes in the ciphertext's length over a span of several carefully crafted connections to discover the desired secret data. Due to this information leakage, the protocol-level compression was abandoned in TLS 1.3. Although the protocol-level compression failed, the benefits of compressing administrative data remain.

How certificate compression works

After Google's QUIC protocol used certificate compression to improve performance, a discussion emerged about whether something similar could be utilized in TLS. As a result, RFC 8879 was released in 2019 describing an extension that helps reduce the size of certificate chains by compressing them.

The idea is simple: Let both sides, the client and server, define their list of supported compression algorithms. The client sends its list of compression algorithms to the server within a ClientHello extension. The server then compares the lists and chooses a mutually supported algorithm for the compression. The certificates will not be compressed if there is no mutually supported algorithm. After this simple exchange, the server can compress its certificate chain with the negotiated compression algorithm.

The server can also request certificate compression from the client. The only difference is that the server sends its list of supported compression algorithms within a CertificateRequest extension instead of a Hello message.

If a compression algorithm is negotiated, the Certificate message is replaced by a CompressedCertificate message, which looks like this:

16 bits	24 bits	<1 .. 224 - 1> bits
algorithm	uncompressed length	compressed certificate

The first field is the identifier of the compression algorithm used to compress the Certificate message, which the peer uses to decompress the message.

The second field is the length of the uncompressed certificate. This field is used to verify the correctness of the decompression and is a security measure, enforcing a size limit on the certificate before decompressing it.

The last field is the compressed Certificate message.

[ Cheat sheet: Get a list of Linux utilities and commands for managing servers and networks. ]

From a security standpoint, certificate compression does not possess the same vulnerabilities as protocol-level compression. An attacker has no control over individual fragments in the Certificate message because the certificate chains are typically presented on a per-server-name or per-user basis. Although the compression alters the length of the Certificate message, TLS 1.3 provides padding mechanisms to counteract such message length analysis. As a result, attacks like CRIME are not applicable.

Certificate compression in GnuTLS

GnuTLS is among the first libraries to implement the extension for certificate compression. The correctness of basic functionality is ensured by upstream tests, which may also serve as a usage example for users. In addition, thorough solution testing is being developed in the tlsfuzzer project to ensure compliance with RFC 8879. Several bugs have already been discovered through the tlsfuzzer, including the extension being sent in ServerHello or the handshake being dropped after receiving an unknown compression algorithm. The latest upstream release fixes these bugs.

The library provides a simple API to use the certificate compression extension:

int
gnutls_compress_certificate_set_methods(
                        gnutls_session_t session,
		        const gnutls_compression_method_t * methods,
		        size_t methods_len);

The function shown above should be called during the session object setup. It allows users to set the list of supported compression algorithms.

[ Improve your skills managing and using SELinux. ]

Currently, there are three possible compression algorithms: zlib, brotli, and zstd. You set algorithm preferences by placing them earlier in the list, meaning (for example) methods[0] would be preferred over methods[1] by the peer. After setting the supported compression algorithms, it is up to the other peer to decide whether certificates will be compressed depending on its list of supported compression algorithms.

If the peer requests certificate compression by sending the compress_certificate extension, the negotiated compression algorithm can be checked with the following function:

gnutls_compression_method_t
gnutls_compress_certificate_get_selected_method(
                        gnutls_session_t session);

GnuTLS also comes with gnutls-cli, a command-line tool for interaction with the TLS server to test interoperability. Users can enable certificate compression in gnutls-cli by using the --compress-cert=str option, where str stands for one of the algorithm names (zlib, brotli, or zstd). This option can be repeated multiple times, effectively building the list of supported compression algorithms.

Wrap up

Compressing administrative data brings performance benefits but may cause security issues when misused. Protocol-level compression has caused vulnerabilities that allow attackers to discover secret data by observing changes in ciphertext length. On the other hand, certificate compression does not suffer from such attacks.

After seeing the successful utilization of compression in QUIC, the certificate compression was brought into TLS 1.3 as an extension. This extension is documented in RFC 8879 and has recently been implemented in the GnuTLS library. To use certificate compression in GnuTLS, set up the list of supported compression algorithms using the gnutls_compress_certificate_set_methods function. You can also use certificate compression in gnutls-cli by using the --compress-cert option.