Chapter 7. Cache Loaders

Chapter 7. Cache Loaders

7.1. The CacheLoader Interface
7.2. Configuration via XML
7.3. Cache passivation
7.4. CacheLoader use cases
7.4.1. Local cache with store
7.4.2. Replicated caches with all nodes sharing the same store
7.4.3. Replicated caches with only one node having a store
7.4.4. Replicated caches with each node having its own store
7.4.5. Hierarchical caches
7.4.6. TcpDelegatingCacheLoader
7.4.7. RmiDelegatingCacheLoader
7.5. JDBC-based CacheLoader
7.5.1. JDBCCacheLoader configuration

JBoss Cache can use a cache loader to back up the in-memory cache to a backend datastore. If JBoss Cache is configured with a cache loader, then the following features are provided:

Currently, the cache loader API looks similar to the TreeCache API. In the future, they will both implement the same interface. The goal is to be able to form hierarchical cache topologies, where one cache can delegate to another, which in turn may delegate to yet another cache.

As of JBossCache 1.3.0, you can now define several cache loaders, in a chain. The impact is that the cache will look at all of the cache loaders in the order they've been configured, until it finds a valid, non-null element of data. When performing writes, all cache loaders are written to (except if the ignoreModifications element has been set to true for a specific cache loader. See the configuration section below for details.

The cache loader interface is defined in org.jboss.cache.loader.CacheLoader as follows (edited for brevity):

public interface CacheLoader extends Service {

   /**
    * Sets the configuration. Will be called before {@link #create()} and {@link #start()}
    * @param props A set of properties specific to a given CacheLoader
    */
   void setConfig(Properties props);

   void setCache(TreeCache c);


   /**
    * Returns a list of children names, all names are <em>relative</em>. Returns null if 
    * the parent node is not found.
    * The returned set must not be modified, e.g. use Collections.unmodifiableSet(s) to 
    * return the result
    * @param fqn The FQN of the parent
    * @return Set<String>. A list of children. Returns null if no children nodes are present,
    * or the parent is not present
    */
   Set getChildrenNames(Fqn fqn) throws Exception;


   /**
    * Returns the value for a given key. Returns null if the node doesn't exist, or the value 
    * is not bound
    */
   Object get(Fqn name, Object key) throws Exception;


   /**
    * Returns all keys and values from the persistent store, given a fully qualified name.
    * 
    * NOTE that the expected return value of this method has changed from JBossCache 1.2.x 
    * and before!  This will affect cache loaders written prior to JBossCache 1.3.0 and such 
    * implementations should be checked for compliance with the behaviour expected.
    *  
    * @param name
    * @return Map<Object,Object> of keys and values for the given node. Returns null if the 
    * node is not found.  If the node is found but has no attributes, this method returns 
    * an empty Map.
    * @throws Exception
    */
   Map get(Fqn name) throws Exception;



   /**
    * Checks whether the CacheLoader has a node with Fqn
    * @return True if node exists, false otherwise
    */
   boolean exists(Fqn name) throws Exception;


   /**
    * Inserts key and value into the attributes hashmap of the given node. If the node does 
    * not exist, all parent nodes from the root down are created automatically
    */
   void put(Fqn name, Object key, Object value) throws Exception;

   /**
    * Inserts all elements of attributes into the attributes hashmap of the given node, 
    * overwriting existing attributes, but not clearing the existing hashmap before
    * insertion (making it a union of existing and new attributes)
    * If the node does not exist, all parent nodes from the root down are created 
    * automatically
    * @param name The fully qualified name of the node
    * @param attributes A Map of attributes. Can be null
    */
   void put(Fqn name, Map attributes) throws Exception;

   /**
    * Inserts all modifications to the backend store. Overwrite whatever is already in
    * the datastore.
    * @param modifications A List<Modification> of modifications
    * @throws Exception
    */
   void put(List modifications) throws Exception;

   /** Removes the given key and value from the attributes of the given node. 
   * No-op if node doesn't exist */
   void remove(Fqn name, Object key) throws Exception;

   /**
    * Removes the given node. If the node is the root of a subtree, this will recursively 
    * remove all subnodes, depth-first
    */
   void remove(Fqn name) throws Exception;

   /** Removes all attributes from a given node, but doesn't delete the node itself */
   void removeData(Fqn name) throws Exception;


   /**
    * Prepare the modifications. For example, for a DB-based CacheLoader:
    * 
    * Create a local (JDBC) transaction
    * Associate the local transaction with tx (tx is the key)
    * Execute the coresponding SQL statements against the DB (statements derived from 
    *modifications)
    * 
    * For non-transactional CacheLoader (e.g. file-based), this could be a null operation
    * @param tx            The transaction, just used as a hashmap key
    * @param modifications List<Modification>, a list of all modifications within the given 
    * transaction
    * @param one_phase     Persist immediately and (for example) commit the local JDBC 
    * transaction as well. When true, we won't get a {@link #commit(Object)} or 
    * {@link #rollback(Object)} method call later
    */
   void prepare(Object tx, List modifications, boolean one_phase) throws Exception;

   /**
    * Commit the transaction. A DB-based CacheLoader would look up the local JDBC transaction 
    * associated with tx and commit that transaction
    * Non-transactional CacheLoaders could simply write the data that was previously saved 
    * transiently under the given tx key, to (for example) a file system (note this only holds if 
    * the previous prepare() did not define one_phase=true
    */
   void commit(Object tx) throws Exception;

   /**
    * Roll the transaction back. A DB-based CacheLoader would look up the local JDBC 
    * transaction associated with tx and roll back that transaction
    */
   void rollback(Object tx);

   /**
    * Fetch the entire state for this cache from secondary storage (disk, DB) and return
    * it as a byte buffer. This is for initialization of a new cache from a remote cache. 
    * The new cache would then call storeEntireState()
    * todo: define binary format for exchanging state
    */
   byte[] loadEntireState() throws Exception;

   /** Store the given state in secondary storage. Overwrite whatever is currently in storage */
   void storeEntireState(byte[] state) throws Exception;
}

NOTE: the contract defined by the CacheLoader interface has changed from JBoss Cache 1.3.0 onwards, specifically with the get(Fqn fqn) method. Special care must be taken with custom CacheLoader implementations to ensure this new contract is still adhered to. See the javadoc above on this method for details, or visit this wiki page for more discussion on this.

CacheLoader implementations that need to support partial state transfer should also implement the subinterface org.jboss.cache.loader.ExtendedCacheLoader:

public interface ExtendedCacheLoader extends CacheLoader
{
   /**
    * Fetch a portion of the state for this cache from secondary storage 
    * (disk, DB) and return it as a byte buffer.
    * This is for activation of a portion of new cache from a remote cache. 
    * The new cache would then call {@link #storeState(byte[], Fqn)}.
    * 
    * @param subtree Fqn naming the root (i.e. highest level parent) node of
    *                the subtree for which state is requested.
    *                
    * @see org.jboss.cache.TreeCache#activateRegion(String)
    */
   byte[] loadState(Fqn subtree) throws Exception;
   
   /**
    * Store the given portion of the cache tree's state in secondary storage. 
    * Overwrite whatever is currently in secondary storage.  If the transferred 
    * state has Fqns equal to or children of parameter subtree, 
    * then no special behavior is required.  Otherwise, ensure that
    * the state is integrated under the given 'subtree'. Typically
    * in the latter case 'subtree' would be the Fqn of the buddy 
    * backup region for a buddy group; e.g.
    * 
    * If the the transferred state had Fqns starting with "/a" and
    * 'subtree' was "/_BUDDY_BACKUP_/192.168.1.2:5555" then the
    * state should be stored in the local persistent store under
    * "/_BUDDY_BACKUP_/192.168.1.2:5555/a"
    * 
    * @param state   the state to store
    * @param subtree Fqn naming the root (i.e. highest level parent) node of
    *                the subtree included in 'state'.  If the Fqns  
    *                of the data included in 'state' are not 
    *                already children of 'subtree', then their
    *                Fqns should be altered to make them children of 
    *                'subtree' before they are persisted.
    */   
   void storeState(byte[] state, Fqn subtree) throws Exception;
   
   /**
    * Sets the {@link RegionManager} this object should use to manage 
    * marshalling/unmarshalling of different regions using different
    * classloaders.
    *
    * NOTE: This method is only intended to be used by the TreeCache instance 
    * this cache loader is associated with.
    * 
    * @param manager    the region manager to use, or null.
    */
   void setRegionManager(RegionManager manager);

}
   

NOTE: If a cache loader is used along with buddy replication, the cache loader must implement ExtendedCacheLoader unless its FetchPersistentState property is set to false.

NOTE: the contract defined by the ExtendedCacheLoader interface has changed from JBoss Cache 1.4.0 onwards, specifically with the requirement that data passed to storeState method be integrated under the given subtree, even if that data didn't originate in that subtree. This behavior is necessary to properly support buddy replication. Special care must be taken with custom ExtendedCacheLoader implementations to ensure this new contract is still adhered to.



[7] Of course they can enable state transfer, if they want to have a warm or hot cache after startup.