In the Red Hat Enterprise Linux 7.5 Beta, we introduced virtual data optimizer (VDO). VDO is a kernel module that can save disk space and reduce replication bandwidth. VDO sits on top of any block storage device and provides zero-block elimination, deduplication of redundant blocks, and data compression. These are the key phases of the data reduction process that allows VDO to reduce data footprint on storage. VDO applies these phases inline and on-the-fly. Now, lets see what happens in each process (download the beta yourself and try):
- Zero-Block Elimination: In the initial phase, any blocks that consist entirely of zeros are identified and recorded only in metadata. This can be understood with an example of sand mixed in a mug of water. We use filter paper (zero-block elimination) to collect only sand particles (non-zero data blocks) out of water. Similarly, VDO only allows blocks that contain something other than all zeros to filter through to the next phase of processing.
- Deduplication: In this second phase, the incoming data is processed to determined whether it is redundant data (data that has been written before) or not. The redundancy of this data is checked through metadata maintained by the UDS (Universal Deduplication Service) kernel module delivered as part of VDO. Any block of data that is found to be redundant will not be written out. Instead, metadata will be updated to point to the original copy of the block already stored on media.
- Compression: Once the initial zero elimination and deduplication phases are completed, LZ4 compression is applied to the individual data blocks. The compressed data blocks are then packed together into fixed length (4 KB) blocks and stored on media. Because a single physical block can contain many compressed blocks, this can also speed up the performance for reading data off storage.
How to create a VDO volume on a storage device
When VDO creates a volume on a block storage device, it divides the device into 2 portions internally:
- UDS Portion: The size of this portion is fixed unless additional capacity is explicitly specified when the VDO volume is created. It is used store the unique name and location of each block seen as deduplication advice is requested of the UDS driver by the VDO device.
- VDO Portion: The VDO portion is the space that VDO uses to add, delete and modify user data and its associated metadata.
Now, let's create a VDO volume and observe how it interacts with other Linux components.
For this example, I am using a local VM running on KVM Virt-manager on my bare metal machine.
VM Specifications:
OS : RHEL 7.5 pre-release (Since, VDO is integrated in RHEL 7.5)
RAM : 4GB
OS DISK : 20GB
ADDITIONAL DISK : 15GB (VirtIO-blk)
The following are the steps I used to create a VDO volume:
# vdo create --name vdo_vol --device=/dev/vda
This creates the VDO volume on top of my virtio-blk device /dev/vda which can now be accessed as /dev/mapper/vdo_vol.
|
# mkfs.xfs -K /dev/mapper/vdo_vol
This creates an XFS filesystem on top of VDO volume.
|
Note that the "-K" used with the mkfs command speeds up the formatting of XFS file-systems by not sending discard requests at file system creation time. Since the VDO volume has just been created, it is already initialized to zeroes.
# mkdir /vdo_vol
Makes the directory /vdo_vol on which to mount the XFS file system.
|
# mount -o discard /dev/mapper/vdo_vol /vdo_vol
Mounts the file system for general use.
|
The "discard" option while mounting the "/dev/mapper/vdo_vol" is used by the filesystem to inform VDO when blocks have been deleted. This can be done either by using the discard option in the mount command or by periodically running the fstrim utility. Discard behavior is required to free up previously allocated space on thin provisioned devices.
At this point, we have successfully mounted the VDO volume. To confirm this, we observe that the "/dev/mapper/vdo_vol" is mounted on the "/vdo_vol" directory based on the "df -hT" output below:
Observations:
- In this configuration, out of total 15GB disk space, only 12GB is available for user data the remaining 3GB is used for UDS and VDO metadata.
"vda" partitioning in the "lsblk" output is shown below:
|
lsblk output |
Observations:
- Again we can confirm from the vda size, that actually it is 15GB in size, but the user available vdo_vol is providing 12GB of total disk space.
My next blog will look at how much space savings can be achieved with VDO.
Need help ! Drop a comment below .....
Sobre o autor
Navegue por canal
Automação
Últimas novidades em automação de TI para empresas de tecnologia, equipes e ambientes
Inteligência artificial
Descubra as atualizações nas plataformas que proporcionam aos clientes executar suas cargas de trabalho de IA em qualquer ambiente
Nuvem híbrida aberta
Veja como construímos um futuro mais flexível com a nuvem híbrida
Segurança
Veja as últimas novidades sobre como reduzimos riscos em ambientes e tecnologias
Edge computing
Saiba quais são as atualizações nas plataformas que simplificam as operações na borda
Infraestrutura
Saiba o que há de mais recente na plataforma Linux empresarial líder mundial
Aplicações
Conheça nossas soluções desenvolvidas para ajudar você a superar os desafios mais complexos de aplicações
Programas originais
Veja as histórias divertidas de criadores e líderes em tecnologia empresarial
Produtos
- Red Hat Enterprise Linux
- Red Hat OpenShift
- Red Hat Ansible Automation Platform
- Red Hat Cloud Services
- Veja todos os produtos
Ferramentas
- Treinamento e certificação
- Minha conta
- Suporte ao cliente
- Recursos para desenvolvedores
- Encontre um parceiro
- Red Hat Ecosystem Catalog
- Calculadora de valor Red Hat
- Documentação
Experimente, compre, venda
Comunicação
- Contate o setor de vendas
- Fale com o Atendimento ao Cliente
- Contate o setor de treinamento
- Redes sociais
Sobre a Red Hat
A Red Hat é a líder mundial em soluções empresariais open source como Linux, nuvem, containers e Kubernetes. Fornecemos soluções robustas que facilitam o trabalho em diversas plataformas e ambientes, do datacenter principal até a borda da rede.
Selecione um idioma
Red Hat legal and privacy links
- Sobre a Red Hat
- Oportunidades de emprego
- Eventos
- Escritórios
- Fale com a Red Hat
- Blog da Red Hat
- Diversidade, equidade e inclusão
- Cool Stuff Store
- Red Hat Summit