Inscreva-se no feed

As one of our customers pointed out, "job events are not showing in Tower UI", causing significant performance issues for users trying to view job status updates. To make Red Hat Ansible Tower more approachable in viewing Real-Time job status updates, we’ve applied the following performance improvements. 

Performance Improvements

Between the 3.6 and 3.7 releases, there have been significant performance advancements to improve event processing, job running performance and the user interface. This work was done in conjunction with our customers and the Red Hat Scale and Performance team. These include:

  • Added notable performance improvements to event processing to drastically speed up stdout ingestion speed.
  • Updated Ansible Tower to no longer rely on RabbitMQ for clustering and event distribution. Redis is added as a new dependency for event handling.
  • Improved performance in the User Interface for various job views when many simultaneous users are logged into Ansible Tower.
  • Improved job run performance and the write speed of stdout for running playbooks and parallel jobs through optimization of the job dependency/scheduling algorithm.
  • Fixed event processing for inventories with very large numbers of hosts to prevent Ansible Tower slow down.
  • Improved running jobs to no longer block associated inventory updates.

Setup & Performance Tests Covered

To better understand the delay between job run time and job results processing time, as well as running performance benchmarks, we created a HA Tower cluster setup, which is comparable to the average production setup.

Setup

 

Node-A

Node-B

Tower VersionRed Hat Ansible Tower v3.6.2Red Hat Ansible Tower v3.7.1
CPUX5650 @ 2.67GHz 24 CoreX5650 @ 2.67GHz 24 Core
RAM98GB RAM98GB RAM

* VM images are on separate dedicated disks

For proper benchmarking, we needed to have a large number of job events sufficient enough to compare the performance improvements that were made between the versions. To reach that goal, we generated 100k job events by creating an inventory file with 100 fake hosts (with ansible_connection: local) and a job that generated 100 events/hosts. Launching 10 jobs concurrently over a given inventory produced 100k events, which was sufficient for our needs.

Performance Results

Job duration

Tower performance blog 1

The duration of Ansible Tower completing 10 concurrent jobs remains the same on v3.6.2 and v3.7.1,which shows no sudden decrease of performance and thus is sufficient.

Job events processing duration

tower performance blog 2

The duration it takes Ansible Tower to process all of the job results has been improved significantly: a 82.56% speedup of events processing on Ansible Tower v3.7.1 compared to v3.6.2. This is a major improvement and will lead to a drastically improved user experience.

Job events lag

tower performance blog 3

The additional time required to process all the job results after the job finishes has been reduced by 95.77% in Ansible Tower v3.7.1 compared to v3.6.2. Again, a major improvement of performance is clearly visible here, bringing forward a smoother user experience around job events.

Takeaways & Where to go Next 

With performance improvements that are made available starting in Ansible Tower v3.7.1, now Ansible Tower can process all the job results in less additional time to the job run time. This helps users to quickly view the job status updates in the Ansible Tower dashboard in a more Real-Time manner, leading to a much smoother and overall more pleasing user experience.

If you're interested in detailed information on Ansible Tower, then the Ansible Tower Documentation is a must-read. To download and install the latest version, Please visit the Ansible Tower Installation Guide, and to view the release notes of recent Ansible Tower releases, Please visit Release notes 3.7.x and Release notes 3.6.x. If you are interested in more details about the Red Hat Ansible Automation Platform, be sure to check out our e-books.


Sobre os autores

Imaanpreet Kaur joined Red Hat in 2017 and is a software engineer with Red Hat's Performance and Scale Engineering team. Kaur does performance testing, reviews, and checks on Red Hat Satellite and Ansible Tower.

Read full bio

Charan Raj Musali is an Associate Software Engineer with Red Hat’s Performance and Scale Engineering team. 

Read full bio
UI_Icon-Red_Hat-Close-A-Black-RGB

Navegue por canal

automation icon

Automação

Últimas novidades em automação de TI para empresas de tecnologia, equipes e ambientes

AI icon

Inteligência artificial

Descubra as atualizações nas plataformas que proporcionam aos clientes executar suas cargas de trabalho de IA em qualquer ambiente

open hybrid cloud icon

Nuvem híbrida aberta

Veja como construímos um futuro mais flexível com a nuvem híbrida

security icon

Segurança

Veja as últimas novidades sobre como reduzimos riscos em ambientes e tecnologias

edge icon

Edge computing

Saiba quais são as atualizações nas plataformas que simplificam as operações na borda

Infrastructure icon

Infraestrutura

Saiba o que há de mais recente na plataforma Linux empresarial líder mundial

application development icon

Aplicações

Conheça nossas soluções desenvolvidas para ajudar você a superar os desafios mais complexos de aplicações

Original series icon

Programas originais

Veja as histórias divertidas de criadores e líderes em tecnologia empresarial