One of the problems with computer programming is that the same names are constantly used for different purposes. For example, the term namespace is used in many different ways. I often get confused when people talk about namespaces within Kubernetes. For example, some people hear the term and think of virtual clusters, but when I hear it, I think of the Linux namespaces used with pods and containers. Similarly, image can refer to a VM image, a container image, or an OCI image stored at a container registry.
[ You might also like: 7 fun Linux containers/image transports features ]
But in the container world, there is no more overused term than container.
Recently a user created a Podman issue, expressing their frustration over this terminology:
Unclear terminology: image vs. container
My understanding is, that image is a read-only template, whereas container is a read-write thing. Always. Thus if something is a container, then both podman and buildah consider it as container. When something is an image, then both podman and buildah consider it as image.
I call buildah from --name abc scratch. buildah images and podman images have the same output and they do not consider the new thing to be an image.
podman containers ls does not show anything, so the new thing is from podman perspective not a container.
buildah containers however returns the new thing, so that the new thing is a container (with CONTAINER NAME=abc , IMAGE NAME=scratch , BUILDER=*=yes IMAGE ID ="").
So the new thing is a container from buildah perspective, but is not a container from podman perspective. This is utterly confusing.
Please elaborate on the difference between container in terms of buildah and containers in terms of podman.
I think of containers as executing processes within an environment or something that is prepared to be run. In contrast, images are committed containers, which are prepared to be shared with others to create new containers.
The container engines we work with—Podman, Buildah, CRI-O, and Skopeo—all share the same concept of images.
Images are defined in containers/image and stored in different storage or transports, such as container registries, Docker archives, OCI archives, docker-daemon, and containers/storage. I wrote about these storage types or transports in a previous article.
Most people think of images as placed in containers/storage or container registries. For the rest of this description, we will concentrate on containers/storage.
By the way, Skopeo is not really a container engine since it has no concept of containers. Instead, Skopeo just deals with container images and moving them between different types of container storage.
Container/storage containers
The container/storage library also provides its own concept of a storage container. Basically, storage containers are intermediate storage content that has not been committed yet. Think of this as files on disk and some JSON describing the content.
Podman, Buildah, and CRI-O all use storage containers. All three container engines also have additional data specific to themselves.
Buildah containers
Buildah containers include additional content referring to the different commands that make up the Containerfile or Dockerfile.
For example, Workingdir, Env variables, and other data are used to build a container image.
Podman containers
Podman has its own datastore of data related to a Podman container. There is far more data stored in the Podman database than in the Buildah database. You could think of the Buildah database as a subset of the Podman database.
CRI-O containers
CRI-O also has its own datastore to describe its containers.
All three tools have evolved differently and have different ideas and requirements for their own containers.
For example, CRI-O containers evolved with a reliance on a single daemon controlling them. The emphasis is on performance and the need to share hundreds/thousands of responses to the Kubernetes CRI per second. Since CRI-O knows it is the only one dealing with the datastore, it can take advantage of storing the information in memory. CRI-O does not need to be concerned with other processes coming in and changing the containers without its knowledge.
Podman, on the other hand, needs to deal with multiple users of its containers at the same time. It needs to rely more on file system locking and ensuring hundreds of Podman executables can reliably share the same datastore. We have talked about eventually merging Podman's datastore into CRI-O so that CRI-O and Podman could work better together, but over time, we feel that the risk/benefit is hard to justify the merge.
Buildah was also developed as an independent tool. The maintainers have pushed back on taking on the unneeded weight and complexity of the Podman datastore for little to no additional benefit. Buildah containers have one purpose: Building a container image, whereas most Podman containers are more about running applications and services. Buildah containers do not contain all of the information that Podman would require.
How would Podman handle a container created with the buildah from scratch
command?
We would still need to treat these partially created containers differently. Therefore having Podman see them as equivalent or even listed by default in podman ps
command would lead users to confusion.
Compatibility
Podman can actually work with these containers.
The latest versions of Podman can now list the storage containers available on the system:
$ podman ps -a --external | grep buildah
578edf9430ee scratch buildah 13 days ago storage working-container
You can mount and unmount these containers:
# podman mount working-container
/home/dwalsh/.local/share/containers/storage/overlay/a4f596beaabdc78efc7694a67d690097e327aa12bbc59165d011e62b646e93c0/merged
# podman unmount working-container
working-container
You can also remove these containers:
$ podman rm working-container
working-container
You can also create these containers using podman build
. Of course, these containers are only temporarily created during the build. Since these Buildah containers don't have the same data as a Podman container, Podman can not start and stop them, and podman ps
does not show them when they are running.
Podman also has a similar ability to work with CRI-O containers.
[ Learn the basics of using Kubernetes in this free cheat sheet. ]
Wrap up
When it comes to the term container, the context is often critical. Understanding the difference between the uses is essential. When it comes to our container tools, we share most of the underlying content, storage, and libraries. But there are legitimate reasons why each tool has slightly different concepts or definitions or their containers.
About the author
Daniel Walsh has worked in the computer security field for over 30 years. Dan is a Senior Distinguished Engineer at Red Hat. He joined Red Hat in August 2001. Dan leads the Red Hat Container Engineering team since August 2013, but has been working on container technology for several years.
Dan helped developed sVirt, Secure Virtualization as well as the SELinux Sandbox back in RHEL6 an early desktop container tool. Previously, Dan worked Netect/Bindview's on Vulnerability Assessment Products and at Digital Equipment Corporation working on the Athena Project, AltaVista Firewall/Tunnel (VPN) Products. Dan has a BA in Mathematics from the College of the Holy Cross and a MS in Computer Science from Worcester Polytechnic Institute.
Browse by channel
Automation
The latest on IT automation for tech, teams, and environments
Artificial intelligence
Updates on the platforms that free customers to run AI workloads anywhere
Open hybrid cloud
Explore how we build a more flexible future with hybrid cloud
Security
The latest on how we reduce risks across environments and technologies
Edge computing
Updates on the platforms that simplify operations at the edge
Infrastructure
The latest on the world’s leading enterprise Linux platform
Applications
Inside our solutions to the toughest application challenges
Original shows
Entertaining stories from the makers and leaders in enterprise tech
Products
- Red Hat Enterprise Linux
- Red Hat OpenShift
- Red Hat Ansible Automation Platform
- Cloud services
- See all products
Tools
- Training and certification
- My account
- Customer support
- Developer resources
- Find a partner
- Red Hat Ecosystem Catalog
- Red Hat value calculator
- Documentation
Try, buy, & sell
Communicate
About Red Hat
We’re the world’s leading provider of enterprise open source solutions—including Linux, cloud, container, and Kubernetes. We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.
Select a language
Red Hat legal and privacy links
- About Red Hat
- Jobs
- Events
- Locations
- Contact Red Hat
- Red Hat Blog
- Diversity, equity, and inclusion
- Cool Stuff Store
- Red Hat Summit