Automating storage management with RHEL System Roles

table { border: #ddd solid 1px; } td, th { padding: 8px; border: #ddd solid 1px; } td p { font-size: 15px !important; }

In this post:

• Learn how the storage system role can automate the configuration of logical volume manager (LVM) volume groups, logical volumes, and filesystems.

Automation can help increase efficiency, save time, and improve consistency, which is why Red Hat Enterprise Linux (RHEL) includes features that help automate many tasks. RHEL System Roles is a collection of Ansible content included in RHEL to help provide consistent workflows and streamline the execution of many manual tasks.

RHEL contains a storage system role that can automate the configuration of logical volume manager (LVM) volume groups, logical volumes, and filesystems. The storage role supports creating filesystems on unpartitioned disks without the use of LVM as well. In addition, the storage role can automate advanced storage functionality such as encryption, deduplication, compression, and RAID.  

The “Managing local storage using RHEL System Roles” documentation includes information and example playbooks covering how to use the storage system role in many different scenarios.  

This post will focus on using the storage role across systems where the storage device names might not always be consistent. For example, if you would like to use the storage role to implement new volume groups, logical volumes, and filesystems across many RHEL hosts, and these hosts have inconsistent storage device names.  

Environment overview

In my example environment, I have a control node system named controlnode running RHEL 8.5 and two managed nodes: rhel8-server3, rhel8-server4, both running RHEL 8.5.    

These servers have the following storage devices:

As you can see, each server has two 10GB disks (blue cells in table), and two 15 GB disks (red cells in table), however the device names of these storage devices are not consistent between the servers. 

In this example, I would like to setup the following new volume groups, logical volumes, and filesystems:

• web_vg volume group, using the two 10 GB disks (blue cells in table)

  • With a web_lv logical volume, using all of the space in the volume group, mounted at /web

• database_vg volume group, using the two 15GB disks (red cells in table)

  • With a database_lv logical volume, using 50% of the space in the volume group, mounted at /database

  • With a backup_lv logical volume, using 20% of the space in the volume group, mounted at /backup

  • 30% of the space in the volume group should be left free for future expansion

Normally when using the storage system role, a list of disk devices is supplied to the role. For example, a basic playbook to create the database_vg volume group, logical volumes, and filesystems on rhel8-server3 using the two 15GB disks would contain:

- hosts: rhel8-server3
  vars:
    storage_pools:
      - name: database_vg
        disks: 
          - vdd
          - vdf
        volumes:
          - name: database_lv
            size: 50%
            mount_point: "/database"
            state: present
          - name: backup_lv
            size: 20%
            mount_point: "/backup"
            state: present
  roles:
    - redhat.rhel_system_roles.storage

While this playbook would work on rhel8-server3 where the 15GB disks have the vdd and vdf device names, it would fail on rhel8-server4 as there is not a vdf device on this host. In addition, while the vdd device does exist on rhel8-server4, it is 10GB, and I wanted this volume group placed on the 15GB disks. 

It would be possible to create a host_vars directory, and define variables that are specific to each host. This would allow me to specify which disks should be used on each host. However if you have more than a few hosts, manually gathering the disk information from each host and specifying it in a host_vars file for each host quickly becomes impractical. 

In this scenario, what I would really like is to have Ansible dynamically find the disks using Ansible facts.

Ansible facts

By default, when Ansible runs a playbook on hosts, the first task will be to gather facts from each host. These facts include a significant amount of information about each host, including information on storage devices.

On my control node, I have an inventory file named inventory.yml with the following entries:

all:
  hosts:
    rhel8-server3:
    rhel8-server4:

$ ansible rhel8-server3 -m setup -i inventory.yml

The gathered facts are displayed, including a list of ansible_devices. In this list, each disk device has an entry, such as:

           "vdc": {
                "holders": [],
                "host": "SCSI storage controller: Red Hat, Inc. Virtio block device (rev 01)",
                "links": {
                    "ids": [],
                    "labels": [],
                    "masters": [],
                    "uuids": []
                },
                "model": null,
                "partitions": {},
                "removable": "0",
                "rotational": "1",
                "sas_address": null,
                "sas_device_handle": null,
                "scheduler_mode": "none",
                "sectors": "20971520",
                "sectorsize": "512",
                "size": "10.00 GB",
                "support_discard": "512",
                "vendor": "0x1af4",
                "virtual": 1
            },

Included in the facts is information on the size of the disk in the size field, as well as information on holders, links, and partitions. 

The holders, links, and partitions fields can be used as an indication to help determine if the disk possibly contains data. As you build a playbook to select the disks that should be used by the storage role, you might want to use these fields to help exclude existing disks that might already contain data. However, this type of logic would not be idempotent, as on the first run the storage role would configure storage on these unused devices. On subsequent runs, the playbook would no longer be able to find unused disks and would fail. 

In the example presented in this blog post, the storage role will control all storage on the systems, except for the disk that the operating systems (OSs) boot from (the vda device on all managed nodes), so I am not concerned about selecting disks that might already contain data.

Note that extreme care must be taken to ensure that you don’t inadvertently identify disks for the storage role to use that contain data, which might result in data loss.  

Using Ansible facts to select disks for storage role use

In this example, I would like to find and use the two 10GB disks for the web_vg volume group, and find and use the two 15GB disks for the database_vg volume group. In my environment, all storage devices start with vd, and my OS is installed on vda on all servers, so I would like to exclude vda when searching for disks to use.

Again, in this environment the storage role is managing all storage on the system other than the vda device, so I am not concerned with the playbook finding and using disks that already contain data. If your environment is not fully managed by the storage role, additional precautions should be taken to ensure the playbook doesn’t use disks that might already contain data, which could result in data loss.   

Based on my environment, I can create a playbook to locate the disks with my criteria:

- hosts: all
  tasks:
    - name: Identify disks that are 10 GB for web_vg
      set_fact:
        web_vg_disks: "{{ ansible_devices | dict2items | selectattr('key', 'match', '^vd.*') | rejectattr('key', 'match', '^vda$') | selectattr('value.size', 'match', '^10.00 GB') | map(attribute='key') | list }}"

    - name: Identify disks that are 15 GB for database_vg
      set_fact:
        database_vg_disks: "{{ ansible_devices | dict2items | selectattr('key', 'match', '^vd.*') | rejectattr('key', 'match', '^vda$') | selectattr('value.size', 'match', '^15.00 GB') | map(attribute='key') | list }}"

    - name: Show value of web_vg_disks
      ansible.builtin.debug:
        var: web_vg_disks

    - name: Show value of database_vg_disks
      ansible.builtin.debug:
        var: database_vg_disks

The first task identifies ansible_devices that start with the device name vd (excluding vda), and identifies the disks that are 10GB in size. These identified disks are assigned to the web_vg_disks list variable. 

The second task works in the same way, identifying 15GB disks for the database_vg volume group and stores the list of identified disks in the database_vg_disks variable. 

The third and forth tasks display the contents of the web_vg_disks and database_vg_disks variables. 

When run, the playbook shows:

TASK [Show value of web_vg_disks] ********************************************************************************************
ok: [rhel8-server3] => {
    "web_vg_disks": [
        "vde",
        "vdc"
    ]
}
ok: [rhel8-server4] => {
    "web_vg_disks": [
        "vdd",
        "vdc"
    ]
}

TASK [Show value of database_vg_disks] ***************************************************************************************
ok: [rhel8-server3] => {
    "database_vg_disks": [
        "vdf",
        "vdd"
    ]
}
ok: [rhel8-server4] => {
    "database_vg_disks": [
        "vdb",
        "vde"
    ]
}

The playbook correctly identified the disks I would like to use on each server.  

Adding storage configuration to the playbook

Now that I have logic to identify the disks I would like to use, the next step is to utilize the storage role to implement my desired volume group, logical volume, and filesystem configuration. 

I’ll add another task to the playbook to call the storage role. The complete playbook, named storage.yml is:

- hosts: all
  tasks:
    - name: Identify disks that are 10 GB for web_vg
      set_fact:
        web_vg_disks: "{{ ansible_devices | dict2items | selectattr('key', 'match', '^vd.*') | rejectattr('key', 'match', '^vda$') | selectattr('value.size', 'match', '^10.00 GB') | map(attribute='key') | list }}"

    - name: Identify disks that are 15 GB for database_vg
      set_fact:
        database_vg_disks: "{{ ansible_devices | dict2items | selectattr('key', 'match', '^vd.*') | rejectattr('key', 'match', '^vda$') | selectattr('value.size', 'match', '^15.00 GB') | map(attribute='key') | list }}"

    - name: Show value of web_vg_disks
      ansible.builtin.debug:
        var: web_vg_disks

    - name: Show value of database_vg_disks
      ansible.builtin.debug:
        var: database_vg_disks

    - name: Run storage role
      vars:
        storage_pools:
          - name: web_vg
            disks: "{{ web_vg_disks }}"
            volumes:
              - name: web_lv
                size: 100%
                mount_point: "/web"
                state: present
          - name: database_vg
            disks: "{{ database_vg_disks }}"
            volumes:
              - name: database_lv
                size: 50%
                mount_point: "/database"
                state: present
              - name: backup_lv
                size: 20%
                mount_point: "/backup"
                state: present
      include_role:
        name: rhel-system-roles.storage

The Run storage role task defines the storage_pool variable to specify my desired storage configuration. It specifies that a web_vg volume group should be created, with a web_lv logical volume utilizing 100% of the space, and with a filesystem mounted at /web. The volume group should use the disks listed in the web_vg_disks variable which the previous task defined based on the discovered disks that met the specified criteria.   

It similarly specifies that the database_vg volume group should be created, with a database_lv logical volume using 50% of the space, and with a filesystem mounted at /database. There should also be a backup_lv logical volume, using 20% of the space, with a filesystem mounted at /backup. The volume group should use the disks listed in the database_vg_disks variable which the previous task defined based on the discovered disks that met the criteria.   

Running the playbook

At this point, everything is in place, and I’m ready to run the playbook. For this demonstration, I’m using a RHEL control node and will run the playbook from the command line. I’ll use the cd command to move into the storage directory, and then use the ansible-playbook command to run the playbook. 

[ansible@controlnode ~]$ cd storage/
[ansible@controlnode storage]$ ansible-playbook storage.yml -b -i inventory.yml

I specify that the storage.yml playbook should be run, that it should escalate to root (the -b flag), and that the inventory.yml file should be used as my Ansible inventory (the -i flag). 

After the playbook completes, I need to verify that there were no failed tasks:

Validating the configuration

To validate the configuration I’ll run the lsblk command on each of the managed nodes:

$ ssh rhel8-server3 lsblk
NAME                      MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0                        11:0    1 1024M  0 rom  
vda                       252:0    0   25G  0 disk 
├─vda1                    252:1    0    1G  0 part /boot
└─vda2                    252:2    0   24G  0 part 
  ├─rhel-root             253:0    0 21.5G  0 lvm  /
  └─rhel-swap             253:1    0  2.5G  0 lvm  [SWAP]
vdb                       252:16   0   20G  0 disk 
vdc                       252:32   0   10G  0 disk 
└─web_vg-web_lv           253:4    0   20G  0 lvm  /web
vdd                       252:48   0   15G  0 disk 
└─database_vg-database_lv 253:3    0   15G  0 lvm  /database
vde                       252:64   0   10G  0 disk 
└─web_vg-web_lv           253:4    0   20G  0 lvm  /web
vdf                       252:80   0   15G  0 disk 
└─database_vg-backup_lv   253:2    0    6G  0 lvm  /backup

$ ssh rhel8-server4 lsblk
NAME                      MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0                        11:0    1 1024M  0 rom  
vda                       252:0    0   20G  0 disk 
├─vda1                    252:1    0    1G  0 part /boot
└─vda2                    252:2    0   19G  0 part 
  ├─rhel-root             253:0    0   17G  0 lvm  /
  └─rhel-swap             253:1    0    2G  0 lvm  [SWAP]
vdb                       252:16   0   15G  0 disk 
└─database_vg-backup_lv   253:2    0    6G  0 lvm  /backup
vdc                       252:32   0   10G  0 disk 
└─web_vg-web_lv           253:4    0   20G  0 lvm  /web
vdd                       252:48   0   10G  0 disk 
└─web_vg-web_lv           253:4    0   20G  0 lvm  /web
vde                       252:64   0   15G  0 disk 
└─database_vg-database_lv 253:3    0   15G  0 lvm  /database

On both servers, I can see that the web_vg volume group was set up on the two 10GB disks, and the database_vg volume group was set up on the two 15GB disks. I can also validate the sizes of the logical volumes match my desired configuration.  

Next steps

This playbook is idempotent, meaning I can re-run it, and the tasks will not change anything again unless needed to bring the system to the desired state.  

On the database_vg, my logical volumes only added up to use 70% of the available space in the volume group (50% went to databale_lv, and 20% went to backup_lv). This means that there is 9GB free in this volume group on each host. If I would like to increase one of these logical volumes and the corresponding filesystem, I can simply go into the playbook and update the percentage of space that should be allocated.  

For example, if I increase the database_lv size from 50% to 60% in the playbook, and then re-run the playbook the database_lv and corresponding /database filesystem will be increased from 15GB to 18GB.  

Conclusion

The storage RHEL System Role can help you quickly and consistently configure storage in your RHEL environment.  

Red Hat offers many RHEL System Roles that can help automate other important aspects of your RHEL environment. To explore additional roles, review the list of available RHEL System Roles and start managing your RHEL servers in a more efficient, consistent and automated manner today.

Want to learn more about the Red Hat Ansible Automation Platform? Check out our e-book, The automation architect's handbook.