JDK approach to address deserialization vulnerability

This article was originally published on the Red Hat Customer Portal. The information may no longer be current.​

Java Deserialization of untrusted data has been a security buzzword for the past couple of years with almost every application using native Java serialization framework being vulnerable to Java deserialization attacks. Since it's inception, there have been many scattered attempts to come up with a solution to best address this flaw. This article focuses on Java deserialization vulnerability and explains how Oracle provides a mitigation framework in it's latest Java Development Kit (JDK) version.

Background

Let's begin by reviewing the Java deserialization process. Java Serialization Framework is JDK's built-in utility that allows Java objects to be converted into byte representation of the object and vice versa. The process of converting Java objects into their binary form is called serialization and the process of reading binary data to construct a Java object is called deserialization. In any enterprise environment the ability to save or retrieve the state of the object is a critical factor in building reliable distributed systems. For instance, a JMS message may be serialized to a stream of bytes and sent over the wire to a JMS destination. A RESTful client application may serialize an OAuth token to disk for future verification. Java's Remote Method Invocation (RMI) uses serialization under the hood to pass objects between JVMs. These are just some of the use cases where Java serialization is used.

Inspecting the Flow

When the application code triggers the deserialization process, ObjectInputStream will be initialized to construct the object from the stream of bytes. ObjectInputStream ensures the object graph that has been serialized is recovered. During this process, ObjectInputStream matches the stream of bytes against the classes that are available in the JVM's classpath.

So, what is the problem?

During deserialization process, when readObject() takes the byte stream to reconstruct the object, it looks for the magic bytes relevant to the object type that has been written to the serialization stream, to determine what object type (e.g. enum, array, String, etc.) it needs to resolve the byte stream to. If the byte stream can not be resolved to one of these types, then it will be resolved to an ordinary object (TC_OBJECT), and finally the local class for that ObjectStreamClass will be retrieved from the JVM's classpath. If the class is not found then an InvalidClassException will be thrown.

The problem arises when readObject() is presented with a byte stream that has been manipulated to leverage classes that have a high chance of being available in the JVM's classpath, also known as gadget classes, and are vulnerable to Remote Code Execution (RCE). So far a number of classes have been identified to be vulnerable to RCE, however research is still ongoing to discover more of such classes. Now you might ask, how these classes can be used for RCE? Well, depending on the nature of the class, the attack can be materialized by constructing the state of that particular class with a malicious payload, which is serialized and is fed at the point in which serialized data is exchanged (i.e. Stream Source) in the above work flow. This tricks JDK to believe this is the trusted byte stream, and it will be deserialized by initializing the class with the payload. Depending on the payload this can have disastrous consequences. Of course the challenge for the adversary is to be able to access the stream source for this purpose, of which the details are outside the scope of this article. A good tool to review for further information on the subject is ysoserial, which is arguably the best tool for generating payloads.

How to mitigate against deserialization?

Loosely speaking, mitigation against a deserialization vulnerability is accomplished by implementing a LookAheadObjectInputStream strategy. The implementation needs to subclass the existing ObjectInputStream to override the resolveClass() method to verify if the class is allowed to be loaded. This approach appears to be an effective way of hardening against deserialization and usually consists of two implementation flavors: whitelist or blacklist. In whitelist approach, implementation only includes the acceptable business classes that are allowed to be deserialized and blocks other classes. Blacklist implementation on the other hand holds a set of well-known classes that are vulnerable and blocks them from being serialized.

Both whitelist and blacklist have their own pros and cons, however, whitelist-based implementation proves to be a better way to mitigate against a deserialization flaw. It effectively follows the principle of checking the input against the good values which have always been a part of security practices. On the other hand, blacklist-based implementation heavily relies on the intelligence gathered around what classes have been vulnerable and gradually include them in the list which is easy enough to be missed or bypassed.

protected Class<?> resolveClass(ObjectStreamClass desc)
                throws IOException, ClassNotFoundException {
      String name = desc.getName();

      if(isBlacklisted(name) ) {
              throw new SecurityException("Deserialization is blocked for security reasons");
      }

      if(isWhitelisted(name) ) {
              throw new SecurityException("Deserialization is blocked for security reasons");
      }

      return super.resolveClass(desc);
}

JDK's new Deserialization Filtering

Although ad hoc implementations exist to harden against a deserialization flaw, the official specification on how to deal with this issue is still lacking. To address this issue, Oracle has recently introduced serialization filtering to improve the security of deserialization of data which seems to have incorporated both whitelist and blacklist scenarios. The new deserialization filtering is targeted for JDK 9, however it has been backported to some of the older versions of JDK as well.

The core mechanism of deserialization filtering is based on an ObjectInputFilter interface which provides a configuration capability so that incoming data streams can be validated during the deserialization process. The status check on the incoming stream is determined by Status.ALLOWED, Status.REJECTED, or Status.UNDECIDED arguments of an enum type within ObjectInputFilter interface. These arguments can be configured depending on the deserialization scenarios, for instance if the intention is to blacklist a class then the argument will return Status.REJECTED for that specific class and allows the rest to be deserialized by returning the Status.UNDECIDED. On the other hand if the intention of the scenario is to whitelist then Status.ALLOWED argument can be returned for classes that match the expected business classes. In addition to that, the filter also allows access to some other information for the incoming deserializing stream, such as the number of array elements when deserializing an array of class (arrayLength), the depth of each nested objects (depth), the current number of object references (references), and the current number of bytes consumed (streamBytes). This information provides more fine-grained assertion points on the incoming stream and return the relevant status that reflects each specific use cases.

Ways to configure the Filter

JDK 9 filtering supports 3 ways of configuring the filter: custom filter, process-wide filter also known as global filter, and built-in filters for the RMI registry and Distributed Garbage Collection (DGC) usage.

Case-based Filters

The configuration scenario for a custom filter occurs when a deserialization requirement is different from any other deserialization process throughout the application. In this use case a custom filter can be created by implementing the ObjectInputFilter interface and override the checkInput(FilterInfo filterInfo) method.

static class VehicleFilter implements ObjectInputFilter {
        final Class<?> clazz = Vehicle.class;
        final long arrayLength = -1L;
        final long totalObjectRefs = 1L;
        final long depth = 1l;
        final long streamBytes = 95L;

        public Status checkInput(FilterInfo filterInfo) {
            if (filterInfo.arrayLength() < this.arrayLength || filterInfo.arrayLength() > this.arrayLength
                    || filterInfo.references() < this.totalObjectRefs || filterInfo.references() > this.totalObjectRefs
                    || filterInfo.depth() < this.depth || filterInfo.depth() > this.depth || filterInfo.streamBytes() < this.streamBytes
                    || filterInfo.streamBytes() > this.streamBytes) {
                return Status.REJECTED;
            }

            if (filterInfo.serialClass() == null) {
                return Status.UNDECIDED;
            }

            if (filterInfo.serialClass() != null && filterInfo.serialClass() == this.clazz) {
                return Status.ALLOWED;
            } else {
                return Status.REJECTED;
            }
        }
    }

JDK 9 has added two methods to the ObjectInputStream class allowing the above filter to be set/get for the current ObjectInputStream:

public class ObjectInputStream
    extends InputStream implements ObjectInput, ObjectStreamConstants {

    private ObjectInputFilter serialFilter;
    ...
    public final ObjectInputFilter getObjectInputFilter() {
        return serialFilter;
    }

    public final void setObjectInputFilter(ObjectInputFilter filter) {
        ...
        this.serialFilter = filter;
    }
    ...
} 

Contrary to JDK 9, latest JDK 8 (1.8.0_144) seems to only allow filter to be set on ObjectInputFilter.Config.setObjectInputFilter(ois, new VehicleFilter()); at the moment.

Process-wide (Global) Filters

Process-wide filter can be configured by setting jdk.serialFilter as either a system property or a security property. If the system property is defined then it is used to configure the filter; otherwise the filter checks for the security property (i.e. jdk1.8.0_144/jre/lib/security/java.security) to configure the filter.

The value of jdk.serialFilter is configured as a sequence of patterns either by checking against the class name or the limits for incoming byte stream properties. Patterns are separated by semicolon and whitespace is also considered to be part of a pattern. Limits are checked before classes regardless of the order in which the pattern sequence is configured. Below are the limit properties which can be used during the configuration:

- maxdepth=value // the maximum depth of a graph
- maxrefs=value // the maximum number of the internal references
- maxbytes=value // the maximum number of bytes in the input stream
- maxarray=value // the maximum array size allowed

Other patterns match the class or package name as returned by Class.getName(). Class/Package patterns accept asterisk (*), double asterisk (**), period (.), and forward slash (/) symbols as well. Below are a couple pattern scenarios that could possibly happens:

// this matches a specific class and rejects the rest
"jdk.serialFilter=org.example.Vehicle;!*" 

 // this matches all classes in the package and all subpackages and rejects the rest 
- "jdk.serialFilter=org.example.**;!*" 

// this matches all classes in the package and rejects the rest 
- "jdk.serialFilter=org.example.*;!*" 

 // this matches any class with the pattern as a prefix
- "jdk.serialFilter=*;

Built-in Filters

JDK 9 has also introduced additional built-in, configurable filters mainly for RMI Registry and Distributed Garbage Collection (DGC) . Built-in filters for RMI Registry and DGC white-list classes that are expected to be used in either of these services. Below are classes for both RMIRegistryImpl and DGCImp:

RMIRegistryImpl

java.lang.Number
java.rmi.Remote
java.lang.reflect.Proxy
sun.rmi.server.UnicastRef
sun.rmi.server.RMIClientSocketFactory
sun.rmi.server.RMIServerSocketFactory
java.rmi.activation.ActivationID
java.rmi.server.UID

DGCImpl

java.rmi.server.ObjID
java.rmi.server.UID
java.rmi.dgc.VMID
java.rmi.dgc.Lease

In addition to these classes, users can also add their own customized filters using sun.rmi.registry.registryFilter and sun.rmi.transport.dgcFilter system or security properties with the property pattern syntax as described in previous section.

Wrapping up

While Java deserialization is not a vulnerability itself, deserialization of untrusted data using JDK's native serialization framework is. It is important to differentiate between the two, as the latter is introduced by a bad application design rather than being a flaw. Java deserialization framework prior to JEP 290 however, did not have any validation mechanism to verify the legitimacy of the objects. While there are a number of ways to mitigate against JDK's lack of assertion on deserializing objects, there is no concrete specification to deal with this flaw within the JDK itself. With JEP 290, Oracle introduced a new filtering mechanism to allow developers to configure the filter based on a number of deserialization scenarios. The new filtering mechanism seems to have made it easier to mitigate against deserialization of untrusted data should the need arises.