Skopeo 1.0 released

We are thrilled to announce that Skopeo 1.0 has been released.

I often talk about all of the new container tools that we have developed over the last few years, and often skim over Skopeo. But Skopeo was the first one, and really has some cool features. 

Skopeo is a tool for moving container images between different types of container storages.  It allows you to copy container images between container registries like docker.io, quay.io, and your internal container registry or different types of storage on your local system. You can copy to a local container/storage repository, even directly into a Docker daemon.  

One of the best things about Skopeo is you don’t have to be root to execute it, and you don’t need to store the images locally if you are just copying from one container registry to another. Skopeo does not require a daemon to be running to perform its operations. I believe it is a much better solution than having to run podman pull IMAGE; podman push IMAGE.

Skopeo is being used all over the world to move container images. It is used in CI/CD systems to keep container registries up to date, as well as loading up container storage for all different kinds of container servers.

History

Skopeo was originally a side project that Red Hat’s Antonio Murdaca started to view the container image JSON file stored on a container registry. Container images, whether they are Open Container Initiative (OCI) images or Docker images consist of two parts. One part is a tarball of the `rootfs` directory.  This directory, which tends to look like the root file system on the linux operating system, contains all of the code and configuration files required to run an application.  

The second part is a JSON file which describes the application, this is the input from the developer of the container image, how he expects the container to be run.  It includes the entrypoint and cmd describing the path to the executable to start the container. It also includes content like environment variables and working directory.  Basically, all of those extra fields people see in the Dockerfile definitions. These fields are not standardized as part of the OCI Image Specification. 

These image tarballs can get very large, I have seen multi-gigabyte images.  The problem was that if you wanted to view the image specification JSON file, the only way to do this was to do a `docker pull IMAGE; docker inspect IMAGE`. A few years ago we opened a pull request with the upstream Docker project to do a `docker inspect --remote IMAGE`, the request was rejected, because the maintainers did not want to complicate the Docker CLI. But we were told that container registries were just web servers, and we should build our own tool to pull down the JSON file. Antonio created that tool and called it Skopeo which is the Greek word for remote viewing.

Antonio figured if he was going to pull the image specification, he might as well pull the image.  Once he pulled the image he figured he could push the image, and Skopeo developed into a tool that can copy container images between all different types of container storage.

Red Hat was working with CoreOS before the companies merged, and CoreOS wanted to use Skopeo to copy images to their hosts for use with their container engine rkt.  But the CoreOS developers did not want to exec out to Skopeo, they wanted to call into a golang library.  This led us to split Skopeo into a command line tool and a separate library github.com/containers/image. This library is now shared by many other container engines including Podman, Buildah, CRI-O.

USAGE

$ skopeo --help

Various operations with container images and container image registries

Usage:
  skopeo [command]

Available Commands:
  copy                                       Copy an IMAGE-NAME from one location to another
  delete                                     Delete image IMAGE-NAME
  help                                       Help about any command
  inspect                                    Inspect image IMAGE-NAME
  list-tags                                  List tags in the transport/repository specified by the REPOSITORY-NAME
  login                                      Login to a container registry
  logout                                     Logout of a container registry
  manifest-digest                            Compute a manifest digest of a file
  standalone-sign                            Create a signature using local files
  standalone-verify                          Verify a signature using local files
  sync                                       Synchronize one or more images from one location to another

Skopeo operates on the following image and repository types:

• Container Registries: 

An image in a registry implementing the "Docker Registry HTTP API V2". docker://docker-reference.  Authorization state is stored in $XDG_RUNTIME_DIR/containers/auth.json, which is set using (skopeo login). 

• Container Storage 

An image located in a local containers/storage image store. Location and image store specified in /etc/containers/storage.conf.

• Local file system  

An existing local directory path storing the manifest, layer tarballs and signatures as individual files. This is a non-standardized format, primarily useful for debugging or noninvasive container inspection.

• Docker Archive

An image is stored in the docker save formatted file. docker-reference is only used when creating such a file, and it must not contain a digest.

• Docker Daemon 

An image docker-reference stored in the docker daemon internal storage. docker-reference must contain either a tag or a digest. Alternatively, when reading images, the format can also be docker-daemon:algo:digest (an image ID).

• Local directory with OCI formatting 

An image tag in a directory compliant with "Open Container Image Layout Specification" at path.

Examples

Inspecting a repository

Examples:

$ skopeo inspect --config docker://quay.io/podman/stable | json_pp
{
   "architecture" : "amd64",
   "config" : {
   "Cmd" : [
      "/bin/bash"
   ],
   "Env" : [
      "DISTTAG=f32container",
      "FGC=f32",
      "container=oci",
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
      "_CONTAINERS_USERNS_CONFIGURED="
   ],
   "Labels" : {
      "license" : "MIT",
      "name" : "fedora",
      "vendor" : "Fedora Project",
      "version" : "32"
   }
   },
   "created" : "2020-05-03T12:27:03.990489916Z",
   "history" : [
                   ...
   ],
   "os" : "linux",
   "rootfs" : {
   "diff_ids" : [
      "sha256:a4c0fa2b217d3fd63d51e55a6fd59432e543d499c0df2b1acd48fbe424f2ddd1",
      "sha256:120e15c4fd15cb42f0fc028a5105f3923b928c3cb766afc6cbc2c14a78b49387",
      "sha256:f100a96598ff0e42eede39a8cd57ff85c3478f942074216572b01d1d614fc083",
      "sha256:dbb194061737e6970cc735cee0b2353d541f51fe12a69cffb3827cce4cdf5c25",
      "sha256:8dc1236d8bfbd7be7b8bf04677f5dd20fa41bbe6bd98688408071b1d4ec3ecf7"
   ],
   "type" : "layers"
   }
}

Copying images

Skopeo can copy container images between various storage mechanisms:

$ skopeo copy docker://registry.access.redhat.com/ubi8-init docker://reg.company.com/ubi-init
Getting image source signatures
Copying blob 58e1deb9693d done  
Copying blob f544909c6b5a done  
Copying blob 78afc5364ad2 done  
Copying config a858c9c7ea done  
Writing manifest to image destination
Storing signatures
a858c9c7ea130b17bad01c858a20f4392085bcc0f25aa5eeee4b16726bed5bab

$ skopeo copy docker://registry.fedoraproject.org/fedora:latest  containers-storage:fedora
Getting image source signatures
Copying blob 3088721d7dbf done  
Copying config d81c91deec done  
Writing manifest to image destination
Storing signatures

Deleting images from a registry

For example,

$ skopeo delete docker://localhost:5000/imagename:latest

Conclusion

Skopeo is a great lightweight tool to help users and administrators maintain their container image infrastructure. Although it has not received the attention that it probably deserves, Skopeo is a really terrific tool to have in your own toolbox. 

The upstream release is now available, we expect to see Skopeo 1.0 in the Red Hat Enterprise Linux 8.3 release.