Transport Layer Security

This article was originally published on the Red Hat Customer Portal. The information may no longer be current.

TLS or Transport Layer Security is one of the most widely used protocols on the Internet.  A replacement for SSL, when you visit a website by typing https:// in your browser, you are most likely using TLS to securely transmit your data to and from the web server. To most people TLS works like magic. This article takes a brief look at TLS internals.
TLS is based on the earlier SSL (Secure Socket Layer) specification developed by Netscape for their Navigator browser. It is an IETF standard protocol first defined in 1999. It is common to use the terms TLS and SSL interchangeably, and sometimes cryptographic literature refers to them collectively as TLS/SSL.

In application design, TLS is usually used to encapsulate application specific protocols such as HTTP, FTP, SMTP, etc. Historically, TLS has been designed to be used with connection-oriented reliable protocols such as TCP (Transmission Control Protocol). However, the newer DTLS (Datagram Transport Layer Security) protocol has been standardized separately and is being used with UDP-based protocols.

The TLS protocol allows a client-server application to communicate securely across an untrusted network (such as the Internet), in a way designed to prevent eavesdropping (stealing information by reading the communication between the sender and receiver) and tampering (changing the transmitted information). It uses asymmetric cryptography for securely exchanging keys between the client and the server, and then uses symmetric cryptography for the actual encryption of secret data being transmitted. Message Authentication Codes (MACs) are used for message integrity.

Most application specific protocols that TLS encapsulates can operate with or without TLS. A lot of websites on the Internet still support HTTP, therefore it is necessary for the client to indicate to the server if it wants to setup a secure (TLS) connection or not. There are usually two ways of doing this:

1. Operate secure (TLS enabled) and non-secure (plain text) services on two separate network ports. A good example is the HTTP protocol. By convention, non-secure (plain text) HTTP uses port 80, while HTTPS (TLS enabled) uses port 443.

2. Use a single network port for both TLS and non-TLS communication. Clients can request the server to switch over to TLS during the initial handshake stage. The protocol extension used for this is usually called "STARTTLS".

Once the client has requested to use TLS for communication, a series of "records" are exchanged between the client and the server. This is called the TLS handshake and is used to establish various parameters which are used for the security of the connection. A simple example of a TLS handshake is presented below:

1. The TLS handshake starts with a negotiation phase. The client sends a "ClientHello" message to the server. This message contains certain parameters, such as the highest TLS version supported by the client, a list of supported cipher suites, and a list of compression methods supported by the client. Newer clients also include the expected server name, to enable virtual hosting of multiple services on the same IP address.
   The server then sends a "ServerHello" message, containing the chosen cipher suite, compression method and TLS version, along with a randomly generated number. The server also sends its certificate to the client.
   The server then sends a ServerHelloDone message indicating that it is done with the initial handshake.
   The client generates a PreMasterSecret, which is encrypted using the public key of the server certificate. It sends this along with a ClientKeyExchange record to the server.
   Finally, the client and the server use the previously exchanged random number and the PreMasterSecret to generate a common secret called the "Master Secret". All other key data is generated by passing this "Master Secret" through Pseudo-Random Number Generators (PRNGs). At this stage the Negotiation phase is complete.

2. The Client now sends a ChangeCipherSpec record to the server, telling it that from now onwards all communication will be encrypted. Finally, the client sends a Finished message to the server, which is encrypted and authenticated.
   The Server decrypts this message using the previously exchanged secrets. (The hash and the MAC.) If it fails to decrypt/verify the message, the connection is torn down.

3. The Server sends a ChangeCipherSpec record to the client and everything described above happens in the reverse direction. At this stage the TLS handshake is complete. 

   

The TLS handshake described above established session keys. These keys are used to encrypt/decrypt data sent between the client and the server using symmetric encryption.
Cipher suites (previously agreed upon during the handshake phase) are used to encrypt the data records.

Once the data transmission is complete, this secure channel is torn down and all the cryptrographic key material is discarded.

Several open-source projects have implemented TLS. OpenSSL, GnuTLS, NSS and JSSE (part of OpenJDK) are some prominent ones.