Applying a Standard Operating Environment with Red Hat Enterprise Linux System Roles and Satellite

Managing large volumes of computer resources is a complex, time-consuming task. Adopting a strategy to standardize the host configuration can reduce management overhead and maintenance complexity. Combined with Red Hat Satellite, Red Hat Enterprise Linux (RHEL) System Roles automates the tasks required to create a standardized operating environment where compute hosts can be configured more consistently at scale.

Configuring hosts to meet specific security or application requirements in a standard operating environment can be tedious and error-prone if you are manually performing these tasks on the command line. Automation with Ansible roles helps to make these configuration tasks deterministic and more scalable to your entire organization.

In this blog, I demonstrate how to configure and apply a standard operating environment using RHEL system roles, and how to apply them automatically with Red Hat Satellite. I also show you that this method for standardizing configuration can be made scalable and repeatable.

What is a standard operating environment?

Defining and applying a standard operating environment (SOE) aims to reduce complexity and automate the management of your compute hosts at scale.

From this article:

“An SOE is a standard operating environment, or a specific computer operating system and collection of software that an IT department defines as a standard build.”

We will define an SOE using multiple RHEL System Roles and automatically apply them to RHEL hosts with Red Hat Satellite.

What are Red Hat Enterprise Linux System Roles?

RHEL System Roles is a collection of Ansible roles and modules that provide a more stable and consistent configuration interface to automate and manage multiple Red Hat Enterprise Linux releases. RHEL System Roles can be combined and applied to RHEL hosts to enforce desired configurations to comply with an SOE. 

You can read more about RHEL System Roles here and here. Brian Smith has written extensively about System Roles here.

What is Red Hat Satellite?

Red Hat Satellite is a tool to manage RHEL hosts on-premises or in the cloud. Organizations can utilize Satellite to manage the entire lifecycle of a RHEL host, from provisioning to updating to upgrading.

Read more about Red Hat Satellite here.

Defining a standard operating environment

It is a common routine to configure the following services: firewall, selinux, ssh daemon, journal logging daemon (logging service for systemd) and timesync. We’ll configure all of these services with RHEL System Roles. 

The RHEL System Roles documentation catalogs the available configuration parameters.

Here is how we’ll configure these services.

We’ll apply these parameters to a Satellite host group in the next section. 

Applying a standard operating environment with Satellite

You would normally apply RHEL System Roles by building an Ansible playbook. In Satellite, we configure a Host Group to automatically apply RHEL System Roles when a host(s) are added to the host group. We define how these RHEL System Roles are configured by applying configuration parameters to the host group.

Prerequisites

1. This blog assumes that RHEL System Roles have been installed and imported into Satellite. See documentation here.
2. RHEL hosts are registered with the Satellite server.

Configure a host group

Satellite host groups are useful for applying configuration attributes to subsets of hosts managed by a Satellite instance. Read more about Satellite host groups here. 

You can create a group through the Web UI by navigating through Configure > Host Groups. Then click on Create Host Group. Brian Smith wrote extensively about setting this up here.

System Role parameters

We’ll use the parameters for each System Role listed below. The Parameter Name comes from the variable documented for each System Role. For example, the firewall System Role documents resetting the firewall to default settings in the yaml below:

---
- name: Reset firewalld example
  hosts: managed-node-01.example.com
  tasks:
  - name: Reset firewalld
    include_role:
      name: rhel-system-roles.firewall

    vars:
      firewall:
        - previous: replaced

In the vars section, we have the variable firewall and the list parameter previous and the attribute or value replaced. 

Firewall

Parameter name: firewall

Type: YAML

Value:

---
port:
- 80/tcp
service:
- cockpit
state: enabled

SElinux

Parameter: selinux_state

Type: YAML

Value: enforcing

SSHD

Parameter: ssh

Type: YAML

Value: 

---
MaxAuthTries: 2
PermitRootLogin: prohibit-password

Journald

Parameter: journald_persistent

Type: YAML

Value: true

Parameter: journald_max_disk_size

Type: YAML

Value: 2048

Parameter: journald_per_user

Type: YAML

Value: true

Parameter: journald_sync_interval

Type: YAML

Value: 1

Timesync

Parameter: timesync_ntp_servers

Type: YAML

Value:

---
- hostname: ca.pool.ntp.org
  pool: yes
  iburst: yes

Click on the Host Group you want to modify. In this case, my host group is named Standard Operating Environment.

To apply the parameters defined above, navigate to the Parameters tab of the desired host group. 

We’ll configure the firewall parameter.

1. Enter the parameter name.
2. Select the type yaml.
3. Enter the parameter values.

Here’s what the parameter configuration will look like in Satellite:

Click Submit to save the configuration.

Add hosts

Add hosts to the host group Standard Operating Environment.

1. Check off the hosts to add to the host group.
2. Click on Select Action.
3. Select Change Group.

In the next menu, select Standard Operating Environment and then click Submit.

The hosts now show that they are in the host group.

Apply the RHEL System Roles against all the hosts in the host group

Navigate to the Host Groups menu.

Apply the Ansible roles.

1. Click on Actions.
2. Click on Run all Ansible roles.

You can then watch the roles being applied.

You can now check that the RHEL System Roles were applied correctly.

Firewall:

$ sudo firewall-cmd --list-all
public (active)
  target: default
  icmp-block-inversion: no
  interfaces: eth0
  sources:
  services: cockpit dhcpv6-client ssh
  ports: 80/tcp
  protocols:
  forward: yes
  masquerade: no
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:

Timesync:

$ sudo cat /etc/chrony.conf
#
# Ansible managed
#

pool ca.pool.ntp.org iburst

# Allow the system clock to be stepped in the first three updates.
makestep 1.0 3

# Enable kernel synchronization of the real-time clock (RTC).
rtcsync

# Record the rate at which the system clock gains/losses time.
driftfile /var/lib/chrony/drift

# Save NTS keys and cookies.
ntsdumpdir /var/lib/chrony

Selinux:

$ sudo getenforce
Enforcing

SSHD:

$ sudo cat /etc/ssh/sshd_config.d/00-ansible_system_role.conf
#
# Ansible managed
#
MaxAuthTries 2
PermitRootLogin prohibit-password

Journald:

$ sudo cat /etc/systemd/journald.conf.d/99-linux-system-roles-journald.conf
# SPDX-License-Identifier: MIT

#
# Ansible managed
#
[Journal]
Storage=persistent
SystemMaxUse=2048M

SplitMode=uid

Scheduling Roles to be applied on a schedule

In the case that someone modifies the configuration of your hosts, Satellite sets the configuration back by re-running the RHEL System Roles on schedule to ensure your hosts conform to your SOE.

From the Host Groups menu, click on the Actions drop-down menu.

Click Configure Ansible Job.

Click on Configure recurring job.

Configure a schedule and click Submit.

What's next?

In this post, I've reviewed how to:

1. Define a standard operating environment
2. Apply an SOE to hosts through Satellite through host groups
3. Schedule a recurring ansible job to re-apply an SOE

Through this configuration, you’ll be able to simply add new hosts to your desired host group to apply the required SOE configuration. In other words, you’ll automatically customize the configuration of your SSH daemon, selinux, firewall, timesync NTP servers, journald logging, or whatever other service you require. All of this is performed with Satellite and RHEL System Roles working together. Finally, Satellite will automatically reapply your SOE configuration to prevent deviations from your required configuration.

You can obtain a no cost, 16 host subscription for Red Hat Enterprise Linux (for personal use) by simply registering for an account at https://console.redhat.com. Find more about the Developer for Individuals subscription here.