Podman is a useful tool for deploying and managing containers. In part one of this article series, I covered how to deploy Podman containers and defined the environment I'll use in the rest of the series. In part two, I demonstrated several ways to list running containers and format their output. Read the previous parts first to understand the environment and necessary toolkit.
This article shows how to use Podman to extract information about the container's external Internet Protocol (IP) addresses.
Podman networking concepts
Before getting the container's IPs, it's important to understand some Podman networking concepts. Podman runs both rootfull and rootless containers, which is a great advantage. However, there are slight differences in how Podman manages rootfull and rootless containers and pods. For a more detailed understanding, I strongly recommend you check the official getting started documentation and tutorials and two other fantastic articles on Podman networking and Podman IP address leasing written by Brent Baude on Enable Sysadmin.
To be succinct and simple, when running rootless containers, the container itself does not have an IP address. Without root privileges, network association is not allowed. Rootless containers make use of the slirp4netns network mode. In contrast, rootfull containers use the Container Network Interface (CNI) plugins and specifically the bridge plugin. This allows them to communicate to each other and the external world using their own IP addresses and a bridged and routed network. They can also use network address translation (NAT).
This article explores both scenarios, starting with the rootfull containers.
Explore a rootfull containers network
First of all, using the commands described in the previous article, list the running rootfull containers and pods:
$ sudo podman ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0158d9f81a96 k8s.gcr.io/pause:3.5 50 minutes ago Up 50 minutes ago 0.0.0.0:8080->80/tcp a28ca9ac0a93-infra
0326028545fa docker.io/library/mysql:latest mysqld 50 minutes ago Up 50 minutes ago 0.0.0.0:8080->80/tcp mysql
62cb7ce5c260 docker.io/library/wordpress:latest apache2-foregroun... 50 minutes ago Up 50 minutes ago 0.0.0.0:8080->80/tcp wordpress
2f988bdf14b5 docker.io/library/httpd:latest httpd-foreground 14 seconds ago Up 14 seconds ago 0.0.0.0:8081->80/tcp httpd
$ sudo podman pod ps --ctr-names
POD ID NAME STATUS CREATED INFRA ID NAMES
a28ca9ac0a93 blog Running 50 minutes ago 0158d9f81a96 a28ca9ac0a93-infra,mysql,wordpress
When Podman is installed, a default network is created for it. List the existing Podman networks:
$ sudo podman network ls
NETWORK ID NAME VERSION PLUGINS
2f259bab93aa podman 0.4.0 bridge,portmap,firewall,tuning
$ sudo podman network inspect podman
[
{
"cniVersion": "0.4.0",
"name": "podman",
"plugins": [
{
"bridge": "cni-podman0",
"hairpinMode": true,
"ipMasq": true,
"ipam": {
"ranges": [
[
{
"gateway": "10.88.0.1",
"subnet": "10.88.0.0/16"
}
]
],
"routes": [
{
"dst": "0.0.0.0/0"
}
],
"type": "host-local"
},
"isGateway": true,
"type": "bridge"
},
{
"capabilities": {
"portMappings": true
},
"type": "portmap"
},
{
"type": "firewall"
},
{
"type": "tuning"
}
]
}
]
Podman's default rootfull network uses the bridge plugin. It is called cni-podman0 and is given a gateway and a subnet. This is the bridge the rootfull containers use to get their external IPs. A veth interface is created for these containers and pod inside a network namespace that is using that bridge, as seen below:
$ sudo nmcli connection show
NAME UUID TYPE DEVICE
Wired connection 1 dbf31db6-f0e9-3b5d-9967-e0ee6b40ba13 ethernet enp1s0
cni-podman0 aa419d98-2ca1-4689-a830-7d163a66fbcd bridge cni-podman0
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:54:00:85:c2:f8 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.30/24 brd 192.168.1.255 scope global noprefixroute enp1s0
valid_lft forever preferred_lft forever
inet6 fe80::e31f:3806:c76d:89cd/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: cni-podman0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 8e:0f:c6:e9:c2:25 brd ff:ff:ff:ff:ff:ff
inet 10.88.0.1/16 brd 10.88.255.255 scope global cni-podman0
valid_lft forever preferred_lft forever
inet6 fe80::8c0f:c6ff:fee9:c225/64 scope link
valid_lft forever preferred_lft forever
5: veth4b13cc0d@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master cni-podman0 state UP group default
link/ether 0e:d1:a3:26:54:4d brd ff:ff:ff:ff:ff:ff link-netns cni-576bb654-3c4e-c8df-5ab4-376b3cc71b6e
inet6 fe80::ec25:e7ff:feb6:14e7/64 scope link
valid_lft forever preferred_lft forever
6: veth63be543d@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master cni-podman0 state UP group default
link/ether 06:4d:d2:1f:1f:fb brd ff:ff:ff:ff:ff:ff link-netns cni-52ee91f6-897d-fa1a-b203-bdbf6dc734f6
inet6 fe80::44d:d2ff:fe1f:1ffb/64 scope link
valid_lft forever preferred_lft forever
Check the bridge configuration for this interface:
$ sudo bridge link list
5: veth4b13cc0d@enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cni-podman0 state forwarding priority 32 cost 2
6: veth63be543d@enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master cni-podman0 state forwarding priority 32 cost 2
$ sudo bridge vlan show
port vlan-id
cni-podman0 1 PVID Egress Untagged
veth4b13cc0d 1 PVID Egress Untagged
veth63be543d 1 PVID Egress Untagged
Now, from inside the network namespaces these interfaces are in, get the IP addresses set for the containers and pods:
$ sudo ip netns list
cni-52ee91f6-897d-fa1a-b203-bdbf6dc734f6 (id: 1)
cni-576bb654-3c4e-c8df-5ab4-376b3cc71b6e (id: 0)
$ sudo ip netns exec cni-52ee91f6-897d-fa1a-b203-bdbf6dc734f6 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether be:4c:f7:f0:47:17 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.88.0.4/16 brd 10.88.255.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::bc4c:f7ff:fef0:4717/64 scope link
valid_lft forever preferred_lft forever
$ sudo ip netns exec cni-576bb654-3c4e-c8df-5ab4-376b3cc71b6e ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 8a:1e:21:dd:fa:91 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.88.0.3/16 brd 10.88.255.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::881e:21ff:fedd:fa91/64 scope link
valid_lft forever preferred_lft forever
But which containers or pods are using each IP? Inspect the containers and pods directly to get this information.
Get external IPs for rootfull containers
The commands to get detailed information about some container or pod are podman inspect and podman pod inspect. Check the podman inspect parameters:
$ podman inspect --help
Display the configuration of object denoted by ID
Description:
Displays the low-level information on an object identified by name or ID.
For more inspection options, see:
podman container inspect
podman image inspect
podman network inspect
podman pod inspect
podman volume inspect
Usage:
podman inspect [options] {CONTAINER|IMAGE|POD|NETWORK|VOLUME} [...]
Examples:
podman inspect fedora
podman inspect --type image fedora
podman inspect CtrID ImgID
podman inspect --format "imageId: {{.Id}} size: {{.Size}}" fedora
Options:
-f, --format string Format the output to a Go template or json (default "json")
-l, --latest Act on the latest container podman is aware of
Not supported with the "--remote" flag
-s, --size Display total file size
-t, --type string Specify inspect-object type ("image", "container" or "all") (default "all")
Get the external IPs and exposed ports for all containers by inspecting them using the following command with some filters to get only these two outputs:
$ sudo podman inspect httpd -f '{{ .NetworkSettings.IPAddress }} {{ .NetworkSettings.Ports }}'
10.88.0.4 map[80/tcp:[{ 8081}]]
$ sudo podman inspect a28ca9ac0a93-infra -f '{{ .NetworkSettings.IPAddress }} {{ .NetworkSettings.Ports }}'
10.88.0.3 map[80/tcp:[{ 8080}]]
$ sudo podman inspect mysql -f '{{ .NetworkSettings.IPAddress }} {{ .NetworkSettings.Ports }}'
10.88.0.3 map[80/tcp:[{ 8080}]]
$ sudo podman inspect wordpress -f '{{ .NetworkSettings.IPAddress }} {{ .NetworkSettings.Ports }}'
10.88.0.3 map[80/tcp:[{ 8080}]]
Since the httpd container is a standalone container, it has its own IP and exposed port. But the a28ca9ac0a93-infra, MySQL, and wordpress containers are part of the same pod, the blog pod. Therefore, they share the same IP address and exposed port, allowing communication from one container to another inside the same pod by using the pod name, the localhost address, or the shared external IP address.
Validate this by making some communication tests between the wordpress and mysql containers, and then from the wordpress container to the standalone httpd container:
$ sudo podman exec -it wordpress /bin/bash
root@blog:/var/www/html# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.88.0.3 blog a28ca9ac0a93-infra
10.88.0.1 host.containers.internal
root@blog:/var/www/html# curl -v telnet://10.88.0.4:80
* Trying 10.88.0.4:80...
* Connected to 10.88.0.4 (10.88.0.4) port 80 (#0)
^C
root@blog:/var/www/html# curl -v telnet://10.88.0.3:3306 --output -
* Trying 10.88.0.3:3306...
* Connected to 10.88.0.3 (10.88.0.3) port 3306 (#0)
J
* RCVD IAC 2
* RCVD IAC 223
^C
root@blog:/var/www/html# curl -v telnet://blog:3306 --output -
* Trying 10.88.0.3:3306...
* Connected to blog (10.88.0.3) port 3306 (#0)
J
* RCVD IAC 2
* RCVD IAC 223
^C
$ sudo podman exec -it httpd /bin/bash
root@2f988bdf14b5:/usr/local/apache2# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.88.0.4 2f988bdf14b5 httpd
10.88.0.1 host.containers.internal
Use the external IP addresses to test communication between the local host where the containers reside and the containers:
$ sudo curl -v http://10.88.0.4
* Trying 10.88.0.4:80...
* Connected to 10.88.0.4 (10.88.0.4) port 80 (#0)
> GET / HTTP/1.1
> Host: 10.88.0.4
> User-Agent: curl/7.79.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Fri, 10 Dec 2021 21:26:34 GMT
< Server: Apache/2.4.51 (Unix)
< Last-Modified: Tue, 07 Dec 2021 21:29:58 GMT
< ETag: "74-5d295133b0ae6"
< Accept-Ranges: bytes
< Content-Length: 116
< Content-Type: text/html
<
<html>
<header>
<title>Enable SysAdmin</title>
</header>
<body>
<p>Hello World!</p>
</body>
</html>
* Connection #0 to host 10.88.0.4 left intact
$ sudo curl -v http://10.88.0.3 | grep "Enable SysAdmin" | head -1
* Trying 10.88.0.3:80...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Connected to 10.88.0.3 (10.88.0.3) port 80 (#0)
> GET / HTTP/1.1
> Host: 10.88.0.3
> User-Agent: curl/7.79.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Fri, 10 Dec 2021 21:29:00 GMT
< Server: Apache/2.4.51 (Debian)
< X-Powered-By: PHP/7.4.26
< Link: <http://192.168.1.30:8080/index.php/wp-json/>; rel="https://api.w.org/"
< Link: <http://192.168.1.30:8080/index.php/wp-json/wp/v2/pages/10>; rel="alternate"; type="application/json"
< Link: <http://192.168.1.30:8080/>; rel=shortlink
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=UTF-8
<
{ [8211 bytes data]
100 21955 0 21955 0 0 462k 0 --:--:-- --:--:-- --:--:-- 476k
* Connection #0 to host 10.88.0.3 left intact
<title>Enable SysAdmin – Just another WordPress site</title>
$ sudo curl -v telnet://10.88.0.3:3306 --output -
* Trying 10.88.0.3:3306...
* Connected to 10.88.0.3 (10.88.0.3) port 3306 (#0)
J
* RCVD IAC 2
* RCVD IAC 223
^C
You could also create your own network configurations that the containers could use to get custom IP addresses from the local network.
Recall that the information demonstrated so far is related to rootfull containers. What about rootless containers?
[ Take a look at 10 considerations for Kubernetes deployments. ]
Explore a rootless containers network
You can use the same images to deploy rootless containers in the same fashion as the rootfull containers, as shown below:
$ podman ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c23ef920ebad docker.io/library/httpd:latest httpd-foreground 54 seconds ago Up 53 seconds ago 0.0.0.0:8081->80/tcp httpd
a07edaeae104 k8s.gcr.io/pause:3.5 13 seconds ago Up 11 seconds ago 0.0.0.0:8080->80/tcp ab3cb4c917bd-infra
f8fbe57cb219 docker.io/library/mysql:latest mysqld 13 seconds ago Up 11 seconds ago 0.0.0.0:8080->80/tcp mysql
43d9972f5220 docker.io/library/wordpress:latest apache2-foregroun... 10 seconds ago Up 8 seconds ago 0.0.0.0:8080->80/tcp wordpress
But a quick look at the network configuration shows a difference:
$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:54:00:85:c2:f8 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.30/24 brd 192.168.1.255 scope global noprefixroute enp1s0
valid_lft forever preferred_lft forever
inet6 fe80::e31f:3806:c76d:89cd/64 scope link noprefixroute
valid_lft forever preferred_lft forever
Rootless containers use a different Podman networking plugin, slirp4netns. Therefore, in order to check the rootless networking information, you must find the containers' network namespace path. Use podman unshare and nsenter to enter these network namespaces, and then check the tap0 interface or virtual device there:
$ ps -ef | grep slirp
localus+ 2333 1 0 13:50 pts/0 00:00:00 /usr/bin/slirp4netns --disable-host-loopback --mtu=65520 --enable-sandbox --enable-seccomp -c -e 3 -r 4 --netns-type=path /run/user/1000/netns/cni-a1f72a2b-46f5-175d-67cc-e914c6e361be tap0
localus+ 2583 1 0 13:50 pts/0 00:00:00 /usr/bin/slirp4netns --disable-host-loopback --mtu=65520 --enable-sandbox --enable-seccomp -c -r 3 --netns-type=path /run/user/1000/netns/rootless-cni-ns tap0
localus+ 4176 1500 0 14:10 pts/0 00:00:00 grep --color=auto slirp
$ podman unshare nsenter --net=/run/user/1000/netns/cni-a1f72a2b-46f5-175d-67cc-e914c6e361be
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: tap0: <BROADCAST,UP,LOWER_UP> mtu 65520 qdisc fq_codel state UNKNOWN group default qlen 1000
link/ether f6:1a:cf:db:97:6b brd ff:ff:ff:ff:ff:ff
inet 10.0.2.100/24 brd 10.0.2.255 scope global tap0
valid_lft forever preferred_lft forever
inet6 fe80::f41a:cfff:fedb:976b/64 scope link
valid_lft forever preferred_lft forever
$ podman unshare nsenter --net=/run/user/1000/netns/rootless-cni-ns
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: tap0: <BROADCAST,UP,LOWER_UP> mtu 65520 qdisc fq_codel state UNKNOWN group default qlen 1000
link/ether 5a:22:95:1e:af:21 brd ff:ff:ff:ff:ff:ff
inet 10.0.2.100/24 brd 10.0.2.255 scope global tap0
valid_lft forever preferred_lft forever
inet6 fe80::5822:95ff:fe1e:af21/64 scope link
valid_lft forever preferred_lft forever
3: cni-podman0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether f2:66:6c:73:26:8b brd ff:ff:ff:ff:ff:ff
inet 10.88.0.1/16 brd 10.88.255.255 scope global cni-podman0
valid_lft forever preferred_lft forever
inet6 fe80::f066:6cff:fe73:268b/64 scope link
valid_lft forever preferred_lft forever
4: veth5fc4083f@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master cni-podman0 state UP group default
link/ether aa:81:ec:b7:61:78 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::a881:ecff:feb7:6178/64 scope link
valid_lft forever preferred_lft forever
The rootless containers have no external IP addresses attached to them. So how can you communicate with them?
Get external IPs for rootless containers
To get a rootless container's IP, use the same process as before but with a different output:
$ podman inspect httpd -f '{{ .NetworkSettings.IPAddress }} {{ .NetworkSettings.Ports }}'
map[80/tcp:[{ 8081}]]
$ podman inspect mysql -f '{{ .NetworkSettings.IPAddress }} {{ .NetworkSettings.Ports }}'
10.88.0.2 map[80/tcp:[{ 8080}]]
$ podman inspect wordpress -f '{{ .NetworkSettings.IPAddress }} {{ .NetworkSettings.Ports }}'
10.88.0.2 map[80/tcp:[{ 8080}]]
As seen above, the httpd standalone container doesn't get an external IP address, but the mysql and wordpress containers get an external IP address because they're inside the same pod, which uses the bridge network by default.
How can you communicate with these rootless containers without an external IP address? Connect by using either the localhost address or the IP address from the host node interface through the containers' exposed ports. Check it out:
$ podman exec -it wordpress /bin/bash
root@blog:/var/www/html# curl -v telnet://10.88.0.2:3306 --output -
* Trying 10.88.0.2:3306...
* Connected to 10.88.0.2 (10.88.0.2) port 3306 (#0)
J
* RCVD IAC 2
* RCVD IAC 223
^C
root@blog:/var/www/html# curl -v telnet://127.0.0.1:3306 --output -
* Trying 127.0.0.1:3306...
* Connected to 127.0.0.1 (127.0.0.1) port 3306 (#0)
J
* RCVD IAC 2
* RCVD IAC 223
^C
root@blog:/var/www/html# curl -v telnet://192.168.1.30:8081
* Trying 192.168.1.30:8081...
* Connected to 192.168.1.30 (192.168.1.30) port 8081 (#0)
^C
It is important to remember whether containers are rootfull or rootless when getting IP address information from the containers so that you know the best way to interact with them and how to configure the networking accordingly.
Optimize communications between applications
Podman offers a network abstraction that makes life much easier for container users, but it is important to understand its concepts to optimize communication between applications. This article shows how to get this information and extract IP addresses and other network information from containers. The next article in this series show how to update container images with Podman.
