Detecting ROBOT and other vulnerabilities using Red Hat testing tools

This article was originally published on the Red Hat Customer Portal. The information may no longer be current.

The TLS (Transport Layer Security) protocol, also known as SSL, underpins the security of most Internet protocols. That means the correctness of its implementations protects the safety of communication across network connections.

The Red Hat Crypto Team, to verify the correctness of the TLS implementations we ship, has created a TLS testing framework which is developed as the open source tlsfuzzer project. That testing framework is being used to detect and fix issues with the OpenSSL, NSS, GnuTLS, and other TLS software we ship.

Recently, Hanno Böck, Juraj Somorovsky, and Craig Young, responsible for discovery of the ROBOT vulnerability, have identified that tlsfuzzer was one of only two tools able to detect the vulnerability at the time they discovered it. This article describes how to use tlsfuzzer to test for two common vulnerabilities - DROWN and ROBOT (which is an extension of the well known Bleichenbacher attack).

Getting tlsfuzzer

tlsfuzzer requires three Python libraries:

• six (PyPI)
• ecdsa (PyPI)
• tlslite-ng (PyPI)

six is available in Red Hat Enterprise Linux 6 and Red Hat Enterprise Linux 7 and can be installed using the following command:

yum install python-six

In Fedora both Python 2 and Python 3 are available, so it needs to be installed using
the following command:

dnf install python2-six

Note: tlsfuzzer and its dependencies are compatible with Python 2.6, Python 2.7, and Python 3, but because Python 2 is the default python on the above mentioned systems, the instructions below will use Python 2 to run it.

Remaining libraries can be downloaded to a single directory and run from there:

git clone https://github.com/tomato42/tlsfuzzer.git                             
cd tlsfuzzer                                                                    
git clone https://github.com/warner/python-ecdsa .python-ecdsa                  
ln -s .python-ecdsa/src/ecdsa/ ecdsa                                            
git clone https://github.com/tomato42/tlslite-ng .tlslite-ng                    
ln -s .tlslite-ng/tlslite/ tlslite

Running tests

The tests that can be run live in scripts/ directory. The test for ROBOT (and Bleichenbacher) is called test-bleichenbacher-workaround.py. The test for DROWN is called test-sslv2-force-cipher.py.

To run those scripts, it's necessary to provide them with the hostname and port the server is running on. For a server running on host example.com on port 443, the commands are as follows:

PYTHONPATH=. python scripts/test-sslv2-force-cipher.py -h example.com -p 443
PYTHONPATH=. python scripts/test-bleichenbacher-workaround.py -h example.com -p 443

If the test finishes with a summary like this:

Test end
successful: 21
failed: 0

It means the server passed the tests successfully (behaves in a standards-compliant way) and likely is not vulnerable.

Note: The server can be vulnerable to the Bleichenbacher attack even if it passes the test (as the attack can use timing of the responses, not only contents or presence). As this script does not measure times of the responses, it cannot detect that covert channel. Passing the test does mean that the attack is much harder to perform, if the server is vulnerable in reality.

Many scripts support additional options that may workaround some peculiarities of the server under test. A listing of them can be obtained by running the script with --help option.

Interpreting failures

Unfortunately, as the tool is primarily aimed at developers, interpreting the errors requires a bit of python knowledge and understanding of TLS. Below is a description of the most common harmless errors that can happen during execution of the script.

Tests in general verify if the server under test is RFC-compliant (does it follow standards like RFC 5246). As the standards are continuously updated to workaround or mitigate known vulnerabilities, standards compliance is a good indicator of overall robustness of the implementation. Fortunately, not all departures from behaviour prescribed in the RFCs are vulnerabilities. It does, however, make testing of such non-compliant implementations harder, and more of a guesswork though.

That being said, some errors in test execution may be a result of unexpected server configuration rather than mismatch between expectation of tlsfuzzer and the server. Read below how to workaround them.

Note: A Failure reported by a script is an indicator of a server not following the expected behaviour, not of failure to communicate. Similarly, a successful test is a test in which server behaved as expected, and does not indicate a successful connection.

General error

When execution of a script encounters an error, it will print a message like this:

zero byte in first byte of random padding ...
Error encountered while processing node <tlsfuzzer.expect.ExpectAlert object at 0x7f96e7e56a90> (child: <tlsfuzzer.expect.ExpectClose object at 0x7f96e7e56ad0>) with last message being: <tlslite.messages.Message object at 0x7f96e79f4090>
Error while processing
Traceback (most recent call last):
  File "scripts/test-bleichenbacher-workaround.py", line 250, in main
    runner.run()
  File "/root/tlsfuzzer/tlsfuzzer/runner.py", line 178, in run
    node.process(self.state, msg)
  File "/root/tlsfuzzer/tlsfuzzer/expect.py", line 571, in process
    raise AssertionError(problem_desc)
AssertionError: Expected alert description "bad_record_mac" does not match received "handshake_failure"

First line indicates the name of the scenario that was run, it can be used to reproduce the run alone (by passing it as the last parameter to the script file, like this: ...workaround.py -h example.com -p 443 "zero byte in first byte of random padding").

Second line indicates at which point in execution the failure happened, in this case during ExpectAlert.

Last line indicates the kind of error condition that was detected, in this case the description of the received alert message didn't match the expected one.

Connection refused or timeout in Connect error Pattern:

Error encountered while processing node <tlsfuzzer.messages.Connect ...
...
    sock.connect((self.hostname, self.port))
  File "/usr/lib64/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
error: [Errno 111] Connection refused

and

Error encountered while processing node <tlsfuzzer.messages.Connect...
...
  File "/usr/lib64/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
timeout: timed out

The hostname or the port are incorrect for the server or some system on-route blocks communication with the server.

Unexpected message - Certificate Request Pattern:

Error encountered while processing node <tlsfuzzer.expect.ExpectServerHelloDone
...
AssertionError: Unexpected message from peer: Handshake(certificate_request)

The server is configured to perform client certificate based authentication and the script does not know how to handle it. The server needs to be reconfigured to not request certificate from the client to perform that test.

Unexpected message - Application Data Pattern:

Error encountered while processing node <tlsfuzzer.expect.ExpectAlert ...
...
AssertionError: Unexpected message from peer: ApplicationData(len=8000)

Note: for most tests it will be ExpectAlert, but in general, the node in question
is the one right after ExpectApplicationData in the script.

In the above mentioned test scripts, that is not an indication of ROBOT or DROWN vulnerability, but it may indicate other issues. The USAGE.md document of the tlsfuzzer project includes more information about interpreting this and other failures.