Red Hat Satellite 6.7 performance improvements

Red Hat Satellite 6.7, included with Red Hat Smart Management, is the next generation Red Hat Enterprise Linux systems management tool and is the combined product of a number of open source projects, including Foreman, Katello, Pulp, Candlepin, and Ansible.  Let’s take a look at the performance and scale improvements of Satellite 6.7. 

Tuning profiler

In Satellite 6.6 we introduced tuning profiles and in 6.7 we're introducing a few new profiles. We want Satellite Server to best meet the needs for your organization’s scale, and with the help of tuning profiler option it is possible to more easily tune the Satellite using one of these profiles: 

• default

• medium

• large

• extra-large

• extra-extra-large

Satellite tuning is set using satellite-installer. For example: If you want to tune the Satellite Server using the medium profile, which is recommended for a Satellite with 5,000 - 10,000 managed hosts, then this command would be used:

satellite-installer --tuning medium 

For more information on tuning sizes, please see our Tuning Guide Section 2.2.

Improvement in Remote Execution (REX)

Red Hat Satellite supports the ability to run jobs with arbitrary commands such as a shell script or Ansible tasks and playbooks.  This ability is known as Remote Execution (REX). 

The Satellite team is focused on improving performance and stability of Red Hat Satellite, and through streamlining the code and addressing some technical debt we were able to significantly improve the performance of remote execution using both SSH and Ansible. Moreover, in Satellite 6.7, it has been tested as well as noticed the significant improvement in remote execution by SSH and Ansible.

• Remote execution via SSH in "6.7" compared to "6.6.2" is 86% faster.

• Remote execution via Ansible in "6.7" results in significant improvement in duration i.e 80% faster as compared to “6.6.2”

Remote Execution (REX)  execution ordering 

Satellite 6.7 has a new feature which allows you to run remote execution jobs on the basis of execution order of hostnames.  For example, this could be an alphabetical order or a random order. Follow these steps to run the jobs: 

• Login to Satellite user interface using your credentials.

• Navigate to Monitor -> Jobs. Then click on Run Job. This may be on the upper right corner, or in the center of the page if you have never run a job before.

• Select the Job Category, template, etc. to configure the job. An example is shown in the screenshot of the Job invocation page.

• Click on "Display advanced fields."

   

• Then choose the option to run the job according to execution order, or a randomized order if preferred.

• At last, click on the submit button.

For more details on this feature, please refer to the feature request.

Upgrading the MongoDB Storage Engine

Starting in Satellite 6.5 you have the option to upgrade the MongoDB storage engine to WiredTiger and we do recommend this for performance reasons.  This upgrade only has to be performed once (If you upgraded to WiredTiger in 6.6 you do not need to do it again in 6.7). For more details about how to upgrade the MongoDB storage engine review the Post-Upgrade Tasks section of the Upgrading and Updating Red Hat Satellite guide.

wiredTiger max cache for mongodb

After upgrading the MongoDB Storage Engine to WiredTiger, under certain circumstances, mongodb consumes randomly high memory (up to 1/2 of all RAM) and this aggressive memory usage limits other processes or can cause OOMkiller to kill mongodb. In order to overcome this situation, tune the cache size by referring the following steps:

Update custom-hiera.yaml:

• Edit /etc/foreman-installer/custom-hiera.yaml and add this entry, inserting the value that is 20% of the physical RAM while keeping in mind the guidelines in this case, approximately 6GB for a 32GB server:

mongodb::server::config_data:
   storage.wiredTiger.engineConfig.cacheSizeGB: 6

• Run the Satellite installer to apply the changes:

# satellite-installer

For more details, please refer to this Kbase article and the tuning guide.

ansible-runner is available

In order to improve Ansible performance and stability in Satellite 6.7, the Red Hat Satellite team made ansible-runner always available for the implementation. In Satellite 6.6, Implementation for running Ansible: ansible-playbook was available. Users could choose between ansible-playbook and ansible-runner as shown in the screenshot from 6.6.          

Satellite 6.6

Satellite 6.7

This change is designed to provide better performance and stability of Ansible within Satellite than what was previously offered by ansible-playbook. For more information please refer to this RFE.

Improved Syncing of Repositories 

Users can synchronize repositories from the Red Hat Content Delivery Network (CDN) to their Satellite, which also applies to synchronizing custom repositories (such as Yum or Puppet) that contain a repository URL. 

Click Content → Sync Status to display the list of available Products. Syncing of repositories in Satellite 6.7 took from 5 minutes to just over 3 minutes compared to Satellite 6.6.2. This has been tested by syncing 20 repositories concurrently. 

Takeaways 

In this post, we discussed the improvements in the Satellite 6.7 as well as how this impacted the performance of the Satellite Server. If you're interested in detailed information on Satellite, then the Red Hat Satellite documentation is a must read.