Containers: Understanding the difference between portability, compatibility and supportability

Portability alone does not offer the entire promise of Linux containers. You also need Compatibility and Supportability. Here’s why:

Portability

The standardized format of containers is what makes them portable between registry servers and container hosts. Since the recent wave of containers are governed by open standards, essentially any container host can push or pull images to and from any container registry. This is foundational to the vision of hybrid and multi cloud.

Red Hat named a Leader in the 2023 Gartner® Magic Quadrant™

Red Hat was positioned highest for ability to execute and furthest for completeness of vision in the Gartner 2023 Magic Quadrant for Container Management.

Read the report

Since the Open Containers Initiative defines the image and distribution specifications, a Container Image can be created with Podman, pushed to any container registry with Skopeo, shared with the world, and consumed by any OCI compatible container engine including Docker, RKT, CRI-O, containerd and, of course Podman or Skopeo. Standardizing on image and distribution formats lets us build infrastructure like registry servers, which can be used to store container images that have many different types of binaries in them including:

• Linux on x86

• Linux on ARM

• Linux on POWER

• Linux on Z

• WindowsMore

In fact, registry servers based on the OCI distribution specification are so popular that there’s a proposal to extend them to include almost any artifact: opencontainers/artifacts. If you wanted to, you could even mix binaries from multiple different hardware architectures or operating systems in the same container image.

Let’s demonstrate that the container image format is the same no matter which type of artifact we pull by pulling a Windows Nano image down on a RHEL 8 container host. Notice that Skopeo can pull the Windows Nano container image, uncompress it, and store it locally even though this is on a RHEL 8 container host:

skopeo --override-os windows copy \
docker://mcr.microsoft.com/windows/nanoserver:1903 \
containers-storage:mcr.microsoft.com/windows/nanoserver:1903

The image is now stored in .local/share/containers/storage/ because it’s running rooteless in this case. You can inspect it with the following command:

skopeo inspect containers-storage:mcr.microsoft.com/windows/nanoserver:1903

Output:

{
    "Name": "mcr.microsoft.com/windows/nanoserver",
    "Digest": "sha256:65d0f8a710e5add87e539b6eb25c724d73fba7c090735133246ff7c5ae1100a5",
    "RepoTags": [],
    "Created": "2020-06-03T11:03:30.541Z",
    "DockerVersion": "",
    "Labels": null,
    "Architecture": "amd64",
    "Os": "windows",
    "Layers": [
        "sha256:c40da4de5e84fba9e2c0f302d2d30570160680b4a2d4ae5a558796337e2dfca2"
    ],
    "Env": null
}

You can also see the image with podman by running the `podman images` command:

$ podman images | grep nano

Output:

mcr.microsoft.com/windows/nanoserver          1903            8b8ec38e0f40   7 days ago     263 MB

Before we try and run this image, let’s discuss compatibility a bit...

Compatibility

This is what determines if the binaries in the container image can actually run on the container host. ARM binaries in a container image will not run on POWER container hosts - Windows Nano binaries in a container image will not run on a RHEL 8 container host. Containers don't offer the ability to run cross-platform binaries, you'd need to use virtualization or emulation for that (example: QEMU System Emulator Targets). This compatibility problem extends to processor architecture, and also versions of the operating system. 

Try running a RHEL 8 container image on a RHEL 4 container host -- that isn't going to work very well. However, as long as the operating systems are reasonably similar, the binaries in the container image will usually run. To demonstrate the limits, let's try and execute this Windows Nano container image on a RHEL 8 container host:

podman run -it --log-level=debug \
containers-storage:mcr.microsoft.com/windows/nanoserver:1903

The first thing you’ll notice is that the image fails to run. Let’s analyze a few key log messages to see why. First, notice that Podman is able to mount the container image. As mentioned before, this is to be expected because container images are platform neutral (they’re just tar files) and governed by open standards, so Linux can uncompress it and mount it: 

DEBU[0000] mounted container "8be871d404f1a15410c63bf7b39f13a87aff4d87fbac5847c526029c802c9131" at "/home/fatherlinux/.local/share/containers/storage/overlay/1449c977519500a8939aaf49d28124c11b113b357922a33e341822e0429477af/merged"

Now, notice the exit error:

DEBU[0000] ExitCode msg: "unable to find user containeruser: no matching entries in passwd file" 
ERRO[0000] unable to find user ContainerUser: no matching entries in passwd file

The ContainerUser is a Windows user which Podman doesn't find in a passwd file in the image. Even if we added an /etc/passwd file, the binaries would fail to run because they are Windows binaries, not Linux ELF binaries. This is a compatibility problem, even though the image is portable and can be copied to a Linux container host.

If you would like to see another example of compatibility problems, even between very similar Linux distros, see The limits of compatibility and supportability with containers. Now, let's discuss how Red Hat manages support of container images...

Supportability

This is what vendors can support. This is about investing in testing, security, performance, architecture, a support team, testing that images and binaries are built in a way that they run correctly on a given set of container hosts  as well as updating the code should security, performance or bugs crop up. 

For example, Red Hat supports RHEL 6, UBI 7, and UBI 8 container images on both RHEL 7 and RHEL 8 container hosts (note that RHEL CoreOS in OpenShift is built from RHEL 8 bits). Red Hat cannot guarantee that every permutation of Linux container image and host combination on the planet will run on RHEL container hosts. Nor can Red Hat go patch and update every Linux container image that’s out there.  

Supportability is about putting a reasonable scope around what is and isn’t covered by a Red Hat subscription. Expanding the scope of container images which are supported, increases the testing, analysis, and repair matrix at a non-linear growth rate. Stated another way, scoping support for container images can be very expensive. 

To demonstrate supportability, let’s run some tests with a container image called Red Hat Universal Base Image 8 (UBI) and a container host based on RHEL 8 Server:

cat /etc/os-release | head -n 4

Output:

NAME="Red Hat Enterprise Linux"
VERSION="8.2 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"

Now, run it in a container:

podman run -it --rm ubi8 cat /etc/os-release | head -n 4

Output:

NAME="Red Hat Enterprise Linux"
VERSION="8.2 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"

This demonstrates a completely supportable image and host combination, which is also compatible, and portable.

Conclusion

If you are running production grade workloads that need reliability over a long lifecycle, run RHEL container images on RHEL container hosts, as this is engineered for portability, compatibility, and scoped for support.

To summarize, with Podman on a RHEL 8 container host, running a UBI 8 container image, you get:

1. Portability - you can move the image where you want. This means you can share infrastructure like registry servers between cloud providers or on premise.

2. Compatibility - the container images and hosts are designed and engineered to work together (See: Engineering compatibility with the Red Hat Universal Base Image). Compatibility is based on hardware architecture (x86 versus ARM), operating system (Linux versus Windows), Linux distribution (RHEL versus other distro), and even age of the Linux distro in the container image - for example, very old images may not work on newer hosts, while very new images may not work on older hosts.

3. Supportability - Red Hat can fix problems in the Container Image, Container Host, Container Engine, and the Linux kernel to make sure that these components work together over a defined life cycle. Supportability is based on a vendor's ability to release, patch, version, and test a set of components together. Also, high quality support is based on a well defined and scoped set of components that are designed and engineered to work together.

So, if you are using Red Hat Enterprise Linux today, and just getting started with containers, check out Red Hat Universal Base Image — it’s portable, compatible, and the most supportable base image available for RHEL and RHEL CoreOS. Learn how to get started with our official documentation.