Subscribe to the feed

In the Red Hat Enterprise Linux 7.5 Beta, we introduced virtual data optimizer (VDO). VDO is a kernel module that can save disk space and reduce replication bandwidth. VDO sits on top of any block storage device and provides zero-block elimination, deduplication of redundant blocks, and data compression. These are the key phases of the data reduction process that allows VDO to reduce data footprint on storage. VDO applies these phases inline and on-the-fly.  Now, lets see what happens in each process (download the beta yourself and try):

  1. Zero-Block Elimination: In the initial phase, any blocks that consist entirely of zeros are identified and recorded only in metadata. This can be understood with an example of sand mixed in a mug of water. We use filter paper (zero-block elimination) to collect only sand particles (non-zero data blocks) out of water.  Similarly, VDO only allows blocks that contain something other than all zeros to filter through to the next phase of processing.
  2. Deduplication: In this second phase, the incoming data is processed to determined whether it is redundant data (data that has been written before) or not. The redundancy of this data is checked through metadata maintained by the UDS (Universal Deduplication Service) kernel module delivered as part of VDO. Any block of data that is found to be redundant will not be written out. Instead, metadata will be updated to point to the original copy of the block already stored on media.  
  3. Compression: Once the initial zero elimination and deduplication phases are completed, LZ4 compression is applied to the individual data blocks. The compressed data blocks are then packed together into fixed length (4 KB) blocks and stored on media.  Because a single physical block can contain many compressed blocks, this can also speed up the performance for reading data off storage.

How to create a VDO volume on a storage device

When VDO creates a volume on a block storage device, it divides the device into 2 portions internally:

  1. UDS Portion: The size of this portion is fixed unless additional capacity is explicitly specified when the VDO volume is created. It is used store the unique name and location of each block seen as deduplication advice is requested of the UDS driver by the VDO device.
  2. VDO Portion: The VDO portion is the space that VDO uses to add, delete and modify user data and its associated metadata.

Now, let's create a VDO volume and observe how it interacts with other Linux components.

For this example, I am using a local VM running on KVM Virt-manager on my bare metal machine.

VM Specifications:

           OS : RHEL 7.5 pre-release (Since, VDO is integrated in RHEL 7.5)

           RAM : 4GB

           OS DISK : 20GB

           ADDITIONAL DISK : 15GB (VirtIO-blk)

The following are the steps I used to create a VDO volume:

# vdo create --name vdo_vol --device=/dev/vda

This creates the VDO volume on top of my virtio-blk device /dev/vda which can now be accessed as /dev/mapper/vdo_vol.

 

 # mkfs.xfs -K /dev/mapper/vdo_vol

This creates an XFS filesystem on top of VDO volume.

 

Note that the "-K" used with the  mkfs command speeds up the formatting of XFS   file-systems by not sending discard requests at file system creation time. Since the VDO volume has just been created, it is already initialized to zeroes.

 # mkdir /vdo_vol    

Makes the directory /vdo_vol on which to mount the XFS file system.

 

# mount -o discard /dev/mapper/vdo_vol /vdo_vol

Mounts the file system for general use.

 

The "discard" option while mounting the "/dev/mapper/vdo_vol" is used by the filesystem to inform VDO when blocks have been deleted. This can be done either by using  the discard option in the mount command or by periodically running the fstrim utility.  Discard behavior is required to free up previously allocated space on thin provisioned devices.

At this point, we have successfully mounted the VDO volume. To confirm this, we observe that the "/dev/mapper/vdo_vol" is mounted on the "/vdo_vol" directory based on  the "df -hT" output below:

 

Observations:

  • In this configuration, out of total 15GB disk space, only 12GB is available for user data the remaining 3GB is used for UDS and VDO metadata.

"vda" partitioning in the "lsblk" output is shown below:

 

lsblk output

Observations:

  • Again we can confirm from the vda size, that actually it is 15GB in size, but the user available vdo_vol is providing 12GB of total disk space.

My next blog will look at how much space savings can be achieved with VDO.

Need help ! Drop a comment below .....

 


저자 소개

채널별 검색

automation icon

오토메이션

기술, 팀, 인프라를 위한 IT 자동화 최신 동향

AI icon

인공지능

고객이 어디서나 AI 워크로드를 실행할 수 있도록 지원하는 플랫폼 업데이트

open hybrid cloud icon

오픈 하이브리드 클라우드

하이브리드 클라우드로 더욱 유연한 미래를 구축하는 방법을 알아보세요

security icon

보안

환경과 기술 전반에 걸쳐 리스크를 감소하는 방법에 대한 최신 정보

edge icon

엣지 컴퓨팅

엣지에서의 운영을 단순화하는 플랫폼 업데이트

Infrastructure icon

인프라

세계적으로 인정받은 기업용 Linux 플랫폼에 대한 최신 정보

application development icon

애플리케이션

복잡한 애플리케이션에 대한 솔루션 더 보기

Original series icon

오리지널 쇼

엔터프라이즈 기술 분야의 제작자와 리더가 전하는 흥미로운 스토리