Some time ago. I needed to write a script that would partition and format disks for a big data project. If the disk size was large enough, I needed to use a GUID Partition Table (GPT) partition, otherwise use a Master Boot Record (MBR) partition. The problem was that I did not have access to systems with large disks to test this script.
I remembered that Red Hat Enterprise Linux (RHEL) 7.5 was shipping with a new feature called the Virtual Disk Optimizer (VDO), which enabled on-disk compression and deduplication. I decided to see if this new feature would help my situation. Could I take a small disk, say 10GB, and make it look like a 10TB disk to the OS?
Obviously, no compression technology would allow me actually store 10TB on a 10GB disk, but in my case, I just needed to logically emulate a 10TB disk. Would this work?
In this example, I use Red Hat Enterprise Linux 8.1 under VirtualBox to test, but this procedure should work with any version of Linux that includes VDO. To accomplish this, I:
- Installed the
[root@defiant ~]# yum install vdo kmod-kvdo
- Created a 10GB disk and attached it to my VM:
[root@defiant ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 32G 0 disk ├─sda1 8:1 0 1G 0 part /boot └─sda2 8:2 0 31G 0 part ├─rhel-root 253:0 0 27.8G 0 lvm / └─rhel-swap 253:1 0 3.2G 0 lvm [SWAP] sdb 8:16 0 10G 0 disk sr0 11:0 1 1024M 0 rom
- Created the VDO volume:
[root@defiant ~]# vdo create --name=vdo-disk --device=/dev/sdb --vdoLogicalSize=10T Creating VDO vdo-disk Starting VDO vdo-disk Starting compression on VDO vdo-disk VDO instance 0 volume is ready at /dev/mapper/vdo-disk
- Created a GPT partition:
[root@defiant ~]# parted /dev/mapper/vdo-disk mklabel gpt mkpart p1 xfs 1MB 10TB Information: You may need to update /etc/fstab.
- Displayed the results:
[root@defiant ~]# parted /dev/mapper/vdo-disk print Model: Linux device-mapper (vdo) (dm) Disk /dev/mapper/vdo-disk: 11.0TB Sector size (logical/physical): 4096B/4096B Partition Table: gpt Disk Flags: Number Start End Size File system Name Flags 1 1049kB 10000GB 10000GB p1
- Created a filesystem and mounted it:
[root@defiant ~]# mkdir /data
- Formatted the disk (this part will probably take a long time):
[root@defiant ~]# mkfs.xfs /dev/mapper/vdo-disk1 meta-data=/dev/mapper/vdo-disk1 isize=512 agcount=10, agsize=268435455 blks = sectsz=4096 attr=2, projid32bit=1 = crc=1 finobt=1, sparse=1, rmapbt=0 = reflink=1 data = bsize=4096 blocks=2441405952, imaxpct=5 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0, ftype=1 log =internal log bsize=4096 blocks=521728, version=2 = sectsz=4096 sunit=1 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0
- Mounted the disk:
[root@defiant ~]# mount /dev/mapper/vdo-disk1 /data [root@defiant ~]# df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 395M 0 395M 0% /dev tmpfs 411M 0 411M 0% /dev/shm tmpfs 411M 5.7M 406M 2% /run tmpfs 411M 0 411M 0% /sys/fs/cgroup /dev/mapper/rhel-root 28G 1.8G 27G 7% / /dev/sda1 1014M 291M 724M 29% /boot tmpfs 83M 0 83M 0% /run/user/1000 /dev/mapper/vdo-disk1 9.1T 65G 9.1T 1% /data
If you followed along, you now have emulated a 10TB disk. Note that, with overhead, it’s only showing 9.1TB, but you can experiment with sizes to get the size you need. Obviously you can’t actually store large amounts of data on this disk, but for the purpose of logically emulating a 10TB disk, this setup has been useful to me.
Want to try out Red Hat Enterprise Linux? Download it now for free.