If you use Ansible, you know the inventory is one of its fundamental pieces. The inventory is just a list of machines and possible variables where you can run your Ansible playbooks.
[ Download now: A system administrator's guide to IT automation. ]
An inventory file can be written in YAML, JSON, or Windows INI format and can describe groups of machines. For example:
---
all:
children:
servers:
hosts:
macmini2:
raspberrypi:
vars:
description: Linux servers for the Nunez family
desktops:
hosts:
dmaf5:
mac-pro-1-1:
vars:
description: Desktops for the Nunez family
You can confirm that the file has the proper structure, too. For example, you might filter only the machines that belong to the desktops
pattern:
[josevnz@dmaf5 ExtendingAnsibleWithPython]$ ansible-inventory --yaml --inventory /home/josevnz/EnableSysadmin/BashHere/hosts.yaml --graph desktops
@desktops:
|--dmaf5
|--mac-pro-1-1
Having a static YAML inventory may not be entirely practical for the following reasons:
- Your host inventory is extensive. Admit it; you have better things to do than edit YAML files, right?
- Your inventory is in a format that is not compatible with Ansible YAML. It may be in a database or plain-text file.
- The servers that are part of your inventory are really dynamic. You create machines on your private cloud as you need them, and their IP addresses change frequently. Or perhaps you have a home network with many roaming devices (tablets and phones). Do you want to maintain that by hand?
Ways to manage inventories in Ansible
There are many ways to manage your inventories in Ansible. Here are some possibilities:
- Convert inventories from legacy formats into Ansible.
- Use dynamic inventories with plugins, specifically Nmap.
- Write your own inventory script to generate inventories dynamically.
- Write an Ansible inventory plugin.
This article goes into the first two items on that list, and follow-up articles will explain the latter two. Remember that whatever method you use, you must accomplish this while following good practices of packaging the tools, using virtual environments, and unit testing the code.
[ Learn more: Ansible vs. Terraform, clarified ]
Don't repeat yourself: Check first if someone wrote it for you
Chances are they did. You can quickly see if someone wrote a plugin that can generate an inventory from a different source like this:
$ ansible-doc -t inventory -l
For example:
$ ansible-doc -t inventory -l
advanced_host_list Parses a 'host list' with ranges
auto Loads and executes an inventory plugin specified in a YAML config
aws_ec2 EC2 inventory source
aws_rds rds instance source
azure_rm Azure Resource Manager inventory plugin
cloudscale cloudscale.ch inventory source
constructed Uses Jinja2 to construct vars and groups based on existing inventory
docker_machine Docker Machine inventory source
docker_swarm Ansible dynamic inventory plugin for Docker swarm nodes
foreman foreman inventory source
gcp_compute Google Cloud Compute Engine inventory source
generator Uses Jinja2 to construct hosts and groups from patterns
gitlab_runners Ansible dynamic inventory plugin for GitLab runners
hcloud Ansible dynamic inventory plugin for the Hetzner Cloud
host_list Parses a 'host list' string
ini Uses an Ansible INI file as inventory source
k8s Kubernetes (K8s) inventory source
kubevirt KubeVirt inventory source
linode Ansible dynamic inventory plugin for Linode
netbox NetBox inventory source
nmap Uses nmap to find hosts to target
online Online inventory source
openshift OpenShift inventory source
openstack OpenStack inventory source
scaleway Scaleway inventory source
script Executes an inventory script that returns JSON
toml Uses a specific TOML file as an inventory source
tower Ansible dynamic inventory plugin for Ansible Tower
virtualbox virtualbox inventory source
vmware_vm_inventory VMware Guest inventory source
vultr Vultr inventory source
yaml Uses a specific YAML file as an inventory source
Use the host_list plugin
This is the simplest plugin. You pass a list of machines or IP addresses, and you're good to go.
[ Need more on Ansible? Take a free technical overview course from Red Hat. Ansible Essentials: Simplicity in Automation Technical Overview. ]
Here's an example using the ping module and the remote user josevnz
:
$ cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.17 mac-pro-1-1
192.168.1.16 macmini2
192.168.1.11 raspberrypi
$ cat /etc/hosts| /bin/cut -f1 -d' '|/bin/grep -P '^[a-z1]'
127.0.0.1
192.168.1.17
192.168.1.16
192.168.1.11
$ ansible -u josevnz -i $(/bin/cat /etc/hosts| /bin/cut -f1 -d' '|/bin/grep -P '^[a-z1]'|/bin/xargs|/bin/sed 's# #,#g') -m ping all
127.0.0.1 | SUCCESS => {
"changed": false,
"ping": "pong"
}
192.168.1.11 | SUCCESS => {
"changed": false,
"ping": "pong"
}
192.168.1.16 | SUCCESS => {
"changed": false,
"ping": "pong"
}
192.168.1.17 | SUCCESS => {
"changed": false,
"ping": "pong"
}
No surprises here. But as you can see, this is not very convenient, as I had to do a little bit of Bash scripting to generate the host list. Also, the inventory is static (in the sense that it comes from the /etc/host
file).
Here is a more interesting plugin that uses Nmap.
Use the Nmap plugin
The Nmap plugin allows you to use the well-known network scanner to build your inventory list. First, I'll explain how this works when you're running Nmap by hand.
[ Download a Bash shell scripting cheat sheet. ]
Crash course on Nmap
You can use Nmap on the command line to get a very good idea of what machines and services are on your network:
$ sudo nmap -v -n -p- -sT -sV -O --osscan-limit --max-os-tries 1 -oX $HOME/home_scan.xml 192.168.1.0/24
Starting Nmap 7.80 ( https://nmap.org ) at 2022-03-05 10:29 EST
NSE: Loaded 45 scripts for scanning.
Initiating ARP Ping Scan at 10:29
Scanning 254 hosts [1 port/host]
Completed ARP Ping Scan at 10:29, 5.10s elapsed (254 total hosts)
Nmap scan report for 192.168.1.0 [host down]
Nmap scan report for 192.168.1.2 [host down]
Initiating Connect Scan at 10:29
Scanning 4 hosts [65535 ports/host]
Discovered open port 443/tcp on 192.168.1.1
Discovered open port 8080/tcp on 192.168.1.1
Discovered open port 445/tcp on 192.168.1.1
Discovered open port 139/tcp on 192.168.1.1
Discovered open port 80/tcp on 192.168.1.1
Discovered open port 80/tcp on 192.168.1.4
Discovered open port 35387/tcp on 192.168.1.4
Keep in mind that this scan is a time-consuming operation. Nmap checks every port and host on your network, so this may take minutes or even hours if you don't tune up your query.
With that in mind, keep these useful links around. You will use them to tune your Nmap arguments:
For this inventory, you care about machines where Ansible can use Secure Shell (SSH) and perform operations. Limiting the port number to TCP 22 will speed up the scanning considerably:
# '-n': 'Never do DNS resolution',
# '-p-': 'All ports. Use -p22 to limit scan to port 22',
# '-sV': 'Probe open ports to determine service/version info',
# '-T4': 'Aggressive timing template',
# '-PE': 'Enable this echo request behavior. Good for internal networks',
# '--version-intensity 1': 'Set version scan intensity. Default is 7',
# '--disable-arp-ping': 'No ARP or ND Ping',
# '--max-hostgroup 100': 'Hostgroup (batch of hosts scanned concurrently) size',
# '--min-parallelism 20': 'Number of probes that may be outstanding for a host group',
# '--osscan-limit': 'Limit OS detection to promising targets',
# '--max-os-tries 1': 'Maximum number of OS detection tries against a target',
# '-oX -': 'Send XML output to STDOUT, avoid creating a temp file'
$ nmap -v -n -p22 -sT -sV --osscan-limit --max-os-tries 1 -oX $HOME/home_scan.xml 192.168.1.0/24
Starting Nmap 7.80 ( https://nmap.org ) at 2022-03-05 10:51 EST
NSE: Loaded 45 scripts for scanning.
Initiating Ping Scan at 10:51
Scanning 256 hosts [2 ports/host]
Completed Ping Scan at 10:51, 2.31s elapsed (256 total hosts)
Nmap scan report for 192.168.1.0 [host down]
Nmap scan report for 192.168.1.2 [host down]
Nmap scan report for 192.168.1.5 [host down]
Nmap scan report for 192.168.1.7 [host down]
...
Completed NSE at 10:51, 0.00s elapsed
Nmap scan report for 192.168.1.1
Host is up (0.0024s latency).
PORT STATE SERVICE VERSION
22/tcp closed ssh
Nmap scan report for 192.168.1.3
Host is up (0.070s latency).
...
Nmap scan report for 192.168.1.11
Host is up (0.00036s latency).
PORT STATE SERVICE VERSION
22/tcp open ssh OpenSSH 8.2p1 Ubuntu 4ubuntu0.4 (Ubuntu Linux; protocol 2.0)
Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel
...
Read data files from: /usr/bin/../share/nmap
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 256 IP addresses (8 hosts up) scanned in 2.71 seconds
But if you don't care about port scanning, then replace -p22
with the -sn
(ping scan) flag:
$ time nmap -v -n -sn --osscan-limit --max-os-tries 1 -oX $HOME/home_scan.xml 192.168.1.0/24
Read data files from: /usr/bin/../share/nmap
Nmap done: 256 IP addresses (8 hosts up) scanned in 2.52 seconds
This small difference may be a factor for bigger networks, but this example keeps the port scanning on port 22.
One more thing: Going forward, I will not use the -n
switch. If it's passed, DNS name resolution is disabled and you won't know the names of the machines you just scanned.
Is it possible to extend Nmap? Yes. Ansible has a nice open source plugin that wraps the Nmap command-line program and parses the results. To illustrate how that works, I want to share a Python script called nmap_scan_rpt.py that can correlate services found by Nmap with security advisories (that's one of the ways you can extend the original tool):
$ git clone git@github.com:josevnz/home_nmap.git $HOME/home_nmap.git
pushd home_nmap/
python3 -m venv $HOME/virtualenv/home_nmap/
. ~/virtualenv/home_nmap/bin/activate
nmap_scan_rpt.py $HOME/home_scan.xml
Feel free to play with the code. You can also run Nmap as a web service, but now I will move back to Ansible with Nmap.
Ansible's Nmap plugin
Now you are ready to explore the Ansible Nmap plugin:
# We do not want to do a port scan, only get the list of hosts dynamically
---
plugin: nmap
address: 192.168.1.0/24
strict: False
ipv4: yes
ports: no
groups:
appliance: "'Amazon' in hostname"
regular: "'host' in hostname"
Then test it:
$ ansible-inventory -i ExtendingAnsibleWithPython/Inventories/home_nmap_inventory.yaml --lis
That produces a nice JSON that Ansible consumes:
{
"_meta": {
"hostvars": {
"android-1c5660ab7065af69.home": {
"ip": "192.168.1.4",
"ports": []
},
"dmaf5.home": {
"ip": "192.168.1.26",
"ports": []
}
},
"all": {
"children": [
"ungrouped"
]
},
"ungrouped": {
"hosts": [
"android-1c5660ab7065af69.home",
"dmaf5.home",
"macmini2",
"new-host-2.home",
"new-host-6.home",
"raspberrypi"
]
}
}
}
NOTE: I could not get the jinja2 'groups' feature to work. Its purpose is to put hosts into dynamic groups based on their hostname.
Dynamic inventories in Ansible
I covered a lot of material, so here is a quick summary of what you have done:
- Introduced the
host_list
plugin and saw some of its obvious limitations, specifically when using a large number of hosts. - Did a quick lesson on Nmap and how to use it to scan all network hosts. You created an inventory using this scan.
- Configured and ran the community Ansible Nmap tool and compared its functionality with a manual Nmap scan.
In the next article in this series, I'll show you how to write your own dynamic inventory script and why this may be a better option than using ready-to-use plugins. And in the final article in the series, I'll explain why it might be better to write an Ansible plugin instead of using an inventory script.
Remember, you can download the code and experiment. The best way to learn is by doing and making mistakes.