Ansible runs on Python, and with Ansible Tower, Python is also the engine under the hood. When working with my customers, I have seen some practical questions that appear to be relatively common. This article provides some tips for setting up and troubleshooting virtualenvs in an Ansible Tower scenario.
1. What is a virtualenv?
A Python virtualenv is basically a directory created with the
virtualenv command (which, as you can guess, is a Python script).
Once this virtualenv is "activated," you can add specific Python modules there and have an isolated environment for experimenting and developing without affecting your main Python/Ansible scripts and playbooks.
Ansible Tower has two virtualenvs under the default directory:
The first is:
awx virtualenv is for the exclusive use of Ansible Tower itself and should not be modified.
The other virtualenv is:
/var/lib/awx/venv/ansible. This virtualenv is what Tower uses to run the templates (playbooks).
Someone could ask: If I need to add Python modules required by my Ansible Tower playbooks, should I add them under the default
Well, you could.
But it is probably a better idea to create another virtualenv if:
- You want to keep the default virtualenv as a stable development environment and want to experiment with new modules in another virtualenv.
- You have different groups of developers working with various projects that require specific Python/Ansible modules.
2. Tips for setting up a virtualenv
I normally follow the Ansible Tower documentation when I am creating a new virtualenv, but there are some topics that I tend to overlook (which later ends with me scratching my head asking, "Hmm... why is this not acting like I was expecting?").
Permissions on the Python modules
Make sure to observe the recommendation to set
umask 0022, not only when you create your virtualenv, but also when you add Python modules to it.
The reason for this is that Ansible Tower runs as the user awx, so this user needs to be able to read the Python modules in the virtualenv.
If the permissions are not set correctly, your templates/playbooks may give you errors like module XYZ is not installed or module XYZ has no attribute ABC when the problem is actually a simple permission issue.
[ Readers also enjoyed: How to create an Ansible Playbook ]
Install Ansible in your virtualenv
The playbooks you want to run using the virtualenv will need your specific Python modules installed in your virtualenv. It makes sense to have the Ansible engine also running from that virtualenv.
You can use the following command to install a specific version of Ansible:
sudo /opt/my-envs/xyz/bin/pip install ansible==2.9.15
Note: If you do not specify the version,
pip installs the latest version. There are behavioral changes between Ansible 2.9 and Ansible 2.10, so it is advisable to check this doc before allowing the installation of 2.10 or a newer version.
Use Pip to list the Python modules
Let's say you want to create a replica of your virtualenv in another environment, or you have multiple Ansible Tower nodes and need to replicate the virtualenv across all nodes. To capture the list of modules for comparison or documentation, you can use:
pip list > pip_list.txt head -5 /tmp/pip_list.txt Package Version ------------- --------- ansible 2.9.15 certifi 2020.12.5 cffi 1.14.4
You can also generate a list in a format that allows the installation of the same module versions somewhere else:
pip freeze > /tmp/pip_freeze.txt head -3 /tmp/pip_freeze.txt ansible==2.9.15 certifi==2020.12.5 cffi==1.14.4
You can then use this freeze file to install the same modules with exactly the same versions using:
sudo /opt/my-envs/xyz/bin/pip install -r pip_freeze.txt
3. Make virtualenvs available in Tower projects/templates
By default, Ansible Tower only knows about its own virtualenvs.
In order to make Tower aware of the additional virtualenvs, you have two options:
- Use REST API calls as described in Ansible Tower documentation
- Use the web interface and navigate to SETTINGS -> SYSTEM:
After doing one of these two steps, you will be able to select the new virtualenvs when you work with Organizations, Projects, and Templates in Tower:
Note: Make sure that the virtualenv directory is readable by the user awx. If it is not, Tower will not list the virtualenv.
"The most important thing to pay attention to is which virtualenv is active and which binaries your playbooks are using."
4. Backing up and restoring a virtualenv
If you take a full Tower backup (using the
setup.sh -b script), the virtualenvs will be automatically backed up there, and you will be able to restore them with
setup.sh -r). Of course, restoring this way means that everything else will also be restored.
If you need to backup/restore the virtualenv, you can always use tools like
tar to do that.
Also, if you saved the output of your
pip freeze command, you can recreate the directory structure and use the
pip install command with the
-r argument as shown above.
Ensure that the restored directory structure is readable by the awx user so that Tower can recognize the virtualenv and list it under the ANSIBLE ENVIRONMENT field in Organizations, Projects, and Templates.
Note: In a scenario with Ansible Tower running on OpenShift, things are different when dealing with virtualenvs. That is a topic for another article.
5. Troubleshooting versions
Which Python version are you using?
One source of potential problems is related to the exact version of Python and Pip used, or the version that the tools are seeing.
Sometimes you need to use the command line to install/verify Python modules, so it is always good to check which Python version you are using.
For example, after activating the virtualenv, check the version:
source /opt/my-envs/xyz/bin/activate (xyz) [admin@control my-envs]$ which python /opt/my-envs/xyz/bin/python (xyz) [admin@control my-envs]$ which python3 /opt/my-envs/xyz/bin/python3 (xyz) [admin@control my-envs]$ python -V Python 3.6.8 (xyz) [admin@control my-envs]$ which pip /opt/my-envs/xyz/bin/pip (xyz) [admin@control my-envs]$ pip3 -V pip 20.2.4 from /opt/my-envs/xyz/lib/python3.6/site-packages/pip (python 3.6)
As you can see in the example above, after activation, all the commands point to paths in my virtualenv, which is what you should expect to see, unless you have other versions in your path, and that could create some problems.
This is particularly important if you do have other Python versions on your server. Sometimes you may want to run the
pip command as the root user, and what root sees as the default Python may not be what your non-root account sees. This would cause the module to be installed under the wrong Python environment, and the templates that you run from Tower would not find the module.
Instead of running
sudo pip install module-name, use the full path for
pip to avoid this problem. Using my previous scenario as an example, that would be:
sudo /opt/my-envs/xyz/bin/pip install module-name
Which Ansible version are you using?
If you still see errors like No module named XYZ, check which Ansible version you are running from the template output:
Does it match the version/location that you have in your virtualenv?
You don't need to install Ansible in your virtualenv, but keep in mind that if Ansible playbooks are executed by a different instance of Ansible, it may not see the Python modules installed in your virtualenv.
[ A free guide from Red Hat: 5 steps to automate your business. ]
Virtualenvs are a great way to create isolated scenarios where you can experiment with different Python/Ansible modules without interfering with other installed modules. Virtualenvs are easy to recreate from scratch. The most important thing to pay attention to is which virtualenv is active and which binaries your playbooks are using.