Understanding the Concepts Behind Virtual Data Optimizer (VDO) in RHEL 7.5 Beta

In the Red Hat Enterprise Linux 7.5 Beta, we introduced virtual data optimizer (VDO). VDO is a kernel module that can save disk space and reduce replication bandwidth. VDO sits on top of any block storage device and provides zero-block elimination, deduplication of redundant blocks, and data compression. These are the key phases of the data reduction process that allows VDO to reduce data footprint on storage. VDO applies these phases inline and on-the-fly.  Now, lets see what happens in each process (download the beta yourself and try):

1. Zero-Block Elimination: In the initial phase, any blocks that consist entirely of zeros are identified and recorded only in metadata. This can be understood with an example of sand mixed in a mug of water. We use filter paper (zero-block elimination) to collect only sand particles (non-zero data blocks) out of water.  Similarly, VDO only allows blocks that contain something other than all zeros to filter through to the next phase of processing.
2. Deduplication: In this second phase, the incoming data is processed to determined whether it is redundant data (data that has been written before) or not. The redundancy of this data is checked through metadata maintained by the UDS (Universal Deduplication Service) kernel module delivered as part of VDO. Any block of data that is found to be redundant will not be written out. Instead, metadata will be updated to point to the original copy of the block already stored on media.  
3. Compression: Once the initial zero elimination and deduplication phases are completed, LZ4 compression is applied to the individual data blocks. The compressed data blocks are then packed together into fixed length (4 KB) blocks and stored on media.  Because a single physical block can contain many compressed blocks, this can also speed up the performance for reading data off storage.

How to create a VDO volume on a storage device

When VDO creates a volume on a block storage device, it divides the device into 2 portions internally:

1. UDS Portion: The size of this portion is fixed unless additional capacity is explicitly specified when the VDO volume is created. It is used store the unique name and location of each block seen as deduplication advice is requested of the UDS driver by the VDO device.
2. VDO Portion: The VDO portion is the space that VDO uses to add, delete and modify user data and its associated metadata.

Now, let's create a VDO volume and observe how it interacts with other Linux components.

For this example, I am using a local VM running on KVM Virt-manager on my bare metal machine.

VM Specifications:

           OS : RHEL 7.5 pre-release (Since, VDO is integrated in RHEL 7.5)

           RAM : 4GB

           OS DISK : 20GB

           ADDITIONAL DISK : 15GB (VirtIO-blk)

The following are the steps I used to create a VDO volume:

# vdo create --name vdo_vol --device=/dev/vda

This creates the VDO volume on top of my virtio-blk device /dev/vda which can now be accessed as /dev/mapper/vdo_vol.

 # mkfs.xfs -K /dev/mapper/vdo_vol

This creates an XFS filesystem on top of VDO volume.

Note that the "-K" used with the  mkfs command speeds up the formatting of XFS   file-systems by not sending discard requests at file system creation time. Since the VDO volume has just been created, it is already initialized to zeroes.

 # mkdir /vdo_vol    

Makes the directory /vdo_vol on which to mount the XFS file system.

# mount -o discard /dev/mapper/vdo_vol /vdo_vol

Mounts the file system for general use.

The "discard" option while mounting the "/dev/mapper/vdo_vol" is used by the filesystem to inform VDO when blocks have been deleted. This can be done either by using  the discard option in the mount command or by periodically running the fstrim utility.  Discard behavior is required to free up previously allocated space on thin provisioned devices.

At this point, we have successfully mounted the VDO volume. To confirm this, we observe that the "/dev/mapper/vdo_vol" is mounted on the "/vdo_vol" directory based on  the "df -hT" output below:

Observations:

• In this configuration, out of total 15GB disk space, only 12GB is available for user data the remaining 3GB is used for UDS and VDO metadata.

"vda" partitioning in the "lsblk" output is shown below:

lsblk output

Observations:

• Again we can confirm from the vda size, that actually it is 15GB in size, but the user available vdo_vol is providing 12GB of total disk space.

My next blog will look at how much space savings can be achieved with VDO.

Need help ! Drop a comment below .....