In part two of our series on Automating Content Management, we created the Red Hat Ansible Automation Platform job templates required to automate the content management in Red Hat Satellite for a large fleet of servers.
We described a simple model:
Update the content
Publish the content
Validate the content
Promote the content to the next life cycle environment and
Rinse and Repeat
Note: This article assumes that you have a functional Red Hat Satellite instance configured for provisioning and an Ansible Automation Control Node (Ansible Tower) instance configured to run Ansible automation plays. Full configuration information or automation code for these environments is beyond the scope of these articles.
Completed descriptions and sample values for all the job template configurations can be found on the Wiki pages of the AutomatingContentManagement repo.
Now that we have prepared all the ingredients, we get to the cooking show moment, where we pull the finished product out!
Creating the publishing workflow
Let’s pull together all of the assets we have created and build the workflow. It takes effort to get here, however, Ansible has made the configuration and testing of all the work much easier and it will save a lot of time in the future. Time to create the workflow.
When creating the Workflow Job Template you will see a new tab in the Web UI for something called the visualizer. The visualizer allows you to “draw” your workflow and connect job templates and define the conditions for execution of the workflow.
In the visualizer, if you hover your mouse over the “START” element, a “+” sign appears, clicking this presents the Add Node dialog box. We are asked to select a Node Type. Select the Job Template type. Our workflow will only contain job templates, however, more sophisticated workflows can take advantage of the other node types. These can be approvals from external sources, inventory or project synchronization, management jobs or even other workflows. Selecting the job templates type presents us with the filtered list of job template assets within our Organization.
We will select the first job in our workflow. I created a naming convention to make this easy to follow and discuss. This is not necessary and in some instances might make the reuse of the job templates confusing, but for our example, it serves two purposes. It groups our elements in order nicely in the UI for easy selection and allows for quickly building our workflow in the proper order. We can select our first node: ACM1PublishContent.
Image 1: Adding the first node in our workflow.
After selecting your job template, click Save. We are taken back to the visualizer.
The result is that we have a single job template added to our workflow. It looks like the following picture. The UI marks our node as a job template.
Image: Visualizing the workflow
Now, let’s add the next node. Hovering over the ACM1PublishContent node will provide another pop-up menu with more elements:
a plus sign icon to add a node to the end of chain.
an information icon that will present the details of the current node.
a pencil icon allowing us to edit the existing node, e.g. to pick a different job template.
a link icon allowing us to link the node to another existing node in the workflow.
and a trashcan icon for deleting the existing node.
We are going to click the plus sign to add the new node. The first thing we will notice is that the new dialog presented asks us to define the conditions under which we want to execute the next node in the workflow. Do we want to run the next node On Success, On Failure or Always?
For our example, we will use the default of On Success and click Next. Then we will select the ACM2PromoteToDev job template. Click Save.
Image 2: Adding nodes to the workflow. Selecting the conditions for execution of the new node.
For any node, we can add multiple nodes that diverge from the “parent” node. These can run based on the exit condition of the parent node. As we will see, nodes can converge as well.
Go ahead and add ACM3CreateTestEnv and ACM4CreateSnaps using the same On Success criterion for execution. When you are finished, you should have something that looks like the following:
Image: The beginning of our content publishing workflow.
This is a pretty straightforward workflow. Simple. Do this task if the previous task succeeded. If a task fails, the workflow simply stops. If you need to add cleanup tasks, you can add a node that executes on failure. For things like logging tasks or email notifications on long-running tasks, you could have additional nodes that run on the “Always” condition.
Note: it is a good idea to periodically save your workflow template using the Save button in the upper right of the screen. Losing your work due to some interruption is a pain.
At this point in our workflow we will diverge to run three sets of tasks in parallel—the build and test tasks for each of our three test environments. As we are creating a Standard Operating Environment on which many of our servers may be based, we want to ensure that the proper selection of systems are tested and can run smoothly on our newly published version. We start by adding all the build job templates to the end of the ACM4CreateSnaps node.
Do this by selecting the plus icon from the ACM4CreateSnaps node each time. Our naming convention follows a Step-dot-SubPath-dot-SubPathStep convention. Select the job templates ACM5.1.1BuildJBoss, ACM5.2.1BuildLAMP, and ACM5.3.1BuildWordPress in turn using the default On Success run type. Add the appropriate Test job template as a node to the end of each Build node using On Success. Your workflow should look something like this:
Image 3: The workflow branches to build and execute all our tests for the content.
Each of the three sets of jobs will run in parallel. The behavior that we want is to publish our content once we know all tests are successful. In this case, we will start by adding ACM6PromoteToQA to the end of one of the test jobs—it doesn’t matter which. We will use the On Success run type.
Once we have one, for the remainder of the test steps we will use the link icon to connect to the existing test node to the ACM6PromoteToQA node. When we have finished linking all the Test nodes to ACM6PromoteToQA there is one more important step. We want to edit the ACM6PromoteToQA node to ensure that we define the “convergence”. By changing it from Any to ALL, we ensure that we will Promote our content only when all tests are successful.
Image 4: Configuring execution criteria for the node where multiple tasks converge.
Click Save. The visualizer should now show that all of the preceding nodes have to return success in order for the promotion to QA job runs.
We can finish the workflow by Adding the ACM7DeleteSnaps job template to the end of ACM6PromoteToQA. We now have the full workflow assembled and the end of the visualization should look like this:
Image 5: The test jobs converge on the “promote to QA” job. All of the test jobs must complete successfully in order for the promotion to occur.
Save your finished workflow.
Running the workflow
You can now launch the workflow using the launch button. As the workflow executes, each of the jobs is called in turn and launches. You will see the jobs progress in a visualizer view of the active workflow. You can also check on the status of the individual jobs in the jobs list. Here, you will see the standard log output of the running Ansible jobs.
Image 6: Workflow starting showing ACM1PublishContent running. Clicking on the running job in the visualizer view brings up the job details window for the active job. The Output tab shows the usual log details.
Image 7: The log output shows the Content and Composite Content Views being published
As with any Ansible job, a workflow path will stop on errors in our example, if an error occurs in node ACM3CreateTestEnv (Creating the test environment), the entire workflow will stop. If an error occurs in one of the paths for building the sample applications, for example at node ACM5.2.1BuildLamp, the build for the LAMP stack fails and that path will stop. The other paths for JBoss and WordPress will continue.
The entire workflow will stop at the convergence at node ACM6PromoteToQA because we have the requirement of ALL for the convergence. All converging steps must succeed. If the convergence was ANY, the workflow would continue to the end, the code would get promoted and the snapshots would be deleted. Definitely not what we want! Code that failed gets promoted and we have no existing system to debug the failure!
Debugging the Ansible workflow is an extension of debugging any Ansible job…working through the templates from the start of the workflow. This can be tedious for long-running tasks like publication.
For the publication step, I have added a variable force_publish that, when set to “true,” ensures that the publication occurs and, when set to “false,” assumes that the content view is current and skips the publication step. This saves a lot of time during testing as publication can take hours depending on the size of the content view.
Image 8: This is what success looks like. January Content views published, tested and promoted!
Remember, completed descriptions and sample values for the job template configurations can be found on the Wiki pages of the Automating Content Management repo.
Maintaining the Content View Definitions
The content view definitions in the content_views.yml file describe the product repositories in the content view, the filters used on those repositories and the life cycle environments to which they should be promoted. For this example, the repositories list is not used. The Satellite administrator manages the content views in the Satellite UI on a daily basis, existing content views and composite content views will continue to be published normally by our code.
As new content is created, new content view elements and composite content view elements can be created in our yaml definition file and we can extend the existing workflow. If new content views require new testing procedures and paths, the entire workflow can easily be copied in the Automation Controller UI and the variables updated to reflect the new conditions. Also extending this model to manage QA content testing and promotion can be easily performed by copying our existing workflow and updating our life cycle targets. Eventually, we can connect multiple workflows together to stage our content for production.
Over the coming months, this project will evolve to support Ansible code that exports content information from Satellite and write the code that will build the content views.
The goal of these articles was to demonstrate the automation of the software content publishing process for a Standard Operating Environment (SOE), including why you would want a SOE and how you would define and create one.
We looked at what the basic process would be like to maintain that SOE over time and how we could reduce our burden using automation. It introduced us to the Ansible collection redhat.satellite and how we use that to drive the automation of Red Hat Satellite 6.9. We looked at Ansible Automation Platform’s new execution environments and how to create them. We reviewed items like using Ansible Vault to manage secrets and some of the standards we might implement for its use in Ansible Automation Platform.
We found that we can back up our precious templates and workflows and even automate their creation in another Automation Controller. Finally, we covered the process of building all of the assets to create a workflow in Ansible Automation Platform and link them together. All of these pull together to build a repeatable, trusted process for delivering OS content to our respective teams in a predictable, timely manner—reducing our task list so we can focus on other critical activities.
I hope that you have enjoyed this content and will return often as we build more automation to manage our software infrastructure. To learn more about what Ansible Automation Platform can do for your environment, read this blog post.
In case you missed any part of the series, check out the previous posts:
About the author
Paul Armstrong helps customers meet business outcomes through an in-depth understanding of complex business problems and by designing, building and documenting complete solutions using Red Hat and open source technologies.