Using Snyk and Podman to scan container images from development to deployment

In this post we’re going to focus on the container scanning functionality available through the Snyk CLI, and how that can integrate with Podman and the new Podman API which landed in Podman 2.x and is available in Red Hat Enterprise Linux 8.3. Using Snyk and the Podman API in this way provides container image scanning directly in your local command line, helping developers and admins to scan images and check for vulnerabilities right from the start of the image development process. 

What's Snyk? Snyk is a developer-first cloud native security platform that provides security insights and remediation throughout the software development lifecycle. Snyk’s integrated service covers dependency scanning for your applications across a variety of languages, security scanning for Terraform, Helm, Kubernetes, and of course container image scanning, with integrations into everything from the IDE, through source code repositories to your CI/CD and orchestration platforms.  

Getting started

The first thing we need to do is install the latest version of Podman.

[matt@localhost ~]$ sudo yum -y install podman

Next, let’s install the Snyk CLI :

[matt@localhost ~]$ sudo yum install npm
[matt@localhost ~]$ sudo npm install -g snyk

You’ll need a free Snyk account for this next bit, so go ahead and sign up here. Once you have your account set up, you can authenticate the Snyk CLI by running :

[matt@localhost ~]$ snyk auth

Firstly, let’s pull an older version of Red Hat’s Universal Base Image (UBI) from the public RedHat registry, which we know has some security vulnerabilities:

[matt@localhost ~]$ podman pull registry.access.redhat.com/ubi7/ubi:7.6
[matt@localhost ~]$ podman images
REPOSITORY                           TAG     IMAGE ID      CREATED        SIZE
registry.access.redhat.com/ubi7/ubi  7.6     247ee58855fd  15 months ago  214 MB

Since Podman is based around open standards and works with OCI-compliant images, Snyk has always been able to scan images created or pulled by Podman, by using Podman to save the image to disk and scanning it from the filesystem. Podman can save images in the Docker archive format, or in OCI archive format, both of which Snyk supports. 

[matt@localhost ~]$ podman save 247ee58855fd -o ubi76.tar
[matt@localhost ~]$ snyk container test docker-archive:ubi76.tar
Testing docker-archive:ubi76.tar...
[ TRUNCATED FOR BREVITY ]
✗ High severity vulnerability found in dbus
  Description: RHSA-2020:2894
  Info: https://snyk.io/vuln/SNYK-RHEL7-DBUS-584292
  Introduced through: dbus@1:1.10.24-13.el7_6
  From: dbus@1:1.10.24-13.el7_6
  Fixed in: 1:1.10.24-14.el7_8

Organization:      matt-jarvis-snyk
Package manager:   rpm
Project name:      docker-image|ubi76.tar
Docker image:      docker-archive:ubi76.tar
Platform:          linux/amd64
Licenses:          enabled

Tested 161 dependencies for known issues, found 68 issues.

Pro tip: use `--file` option to get base image remediation advice.
Example: $ snyk test --docker docker-archive:ubi76.tar --file=path/to/Dockerfile

To remove this message in the future, please run `snyk config set disableSuggestions=true`

Now since this is a public image in an standards compliant registry, if we want to scan this image using Snyk CLI at this point, we can also just do :

[matt@localhost ~]$ snyk container test registry.access.redhat.com/ubi7/ubi:7.6

Snyk has a built-in library for pulling from standards compliant registries, even if it doesn’t find Docker on the host. 

However, if the upstream image is in a private repository which requires a login, this won’t work because Snyk would normally use Docker to handle the registry login. This behaviour is the same even if we log in using Podman, since at this point Snyk doesn’t know how to use Podman to handle this :

[matt@localhost ~]$ podman login registry.redhat.io
[matt@localhost ~]$ snyk container test registry.redhat.io/ubi7/ubi:latest
Please login to the Red Hat Registry using your Customer Portal credentials. Further instructions can be found here: https://access.redhat.com/RegistryAuthentication

Similarly, if the image only exists locally in Podman, this will also not work, as Snyk expects to talk to a local Docker client, and at this point has no knowledge of local images in Podman. As an example, let’s tag our UBI image locally and try to scan it :

[matt@localhost ~]$ podman tag 247ee58855fd mytest
[matt@localhost ~]$ podman images
REPOSITORY                           TAG     IMAGE ID      CREATED        SIZE
registry.access.redhat.com/ubi7/ubi  7.6     247ee58855fd  15 months ago  214 MB
localhost/mytest                     latest  247ee58855fd  15 months ago  214 MB
[matt@localhost ~]$ snyk container test localhost/mytest
connect ECONNREFUSED 127.0.0.1:443

In both those use cases, Snyk would normally use the local Docker client, either through the Registry API or by using the Docker socket. Since there is no Docker binary on the system, here Snyk has attempted to use the Registry API, and failed to connect. By default Snyk doesn’t know those images exist locally in Podman, or how to use Podman to interface with upstream. 

Luckily, Podman provides us with some easy compatibility features through the podman-docker package. This gives us a fake Docker binary, which is basically just a shell wrapper to the Podman binary, and adds a symlink between the Podman API socket and the default location of the Docker socket file.  Out of the box, this package creates a symlink to /var/run/podman/podman.sock, which is the default when running the Podman API as root. 

[matt@localhost ~]$ sudo yum install podman-docker

Now let’s try and scan our local image again :

[matt@localhost ~]$ snyk container test localhost/mytest
connect ENOENT /var/run/docker.sock

Well it still doesn’t work, but this time we’ve got a different error. Snyk has detected a binary called docker, and is now trying to connect to the default location of the privileged Docker socket. Since we are not yet running the Podman API, it can’t connect and so fails again. 

Now as we saw earlier, the podman-docker package creates a symlink from /var/run/podman/podman.sock to /var/run/docker.sock, assuming that the Podman API is going to be run privileged. 

[matt@localhost ~]$ ls -l /var/run/docker.sock 
lrwxrwxrwx. 1 root root 23 Nov  3 13:19 /var/run/docker.sock -> /run/podman/podman.sock

One of the main issues with the Docker socket is that by default it runs privileged, which is considered a security risk in certain enterprise environments. For most local development environments we don’t need to run the Podman API that way, and can run an unprivileged socket somewhere else. We also have a method for communicating that location to Snyk by using the DOCKER_HOST environment variable, which supports both tcp:// and unix:// URL formats.

In a different shell, let’s run the Podman API with an unprivileged socket:

[matt@localhost ~]$ podman system service --time=0 unix://home/matt/podman.sock

This will run foregrounded, so you will have to Ctrl-C the process when finishing the test. Now we need to set the DOCKER_HOST environment variable :

[matt@localhost ~]$ export DOCKER_HOST=unix:///home/matt/podman.sock

And finally let’s try and scan our local image again with Snyk :

[matt@localhost ~]$ snyk container test localhost/mytest

Testing localhost/mytest...

[ TRUNCATED FOR BREVITY ]

✗ High severity vulnerability found in dbus
  Description: RHSA-2020:2894
  Info: https://snyk.io/vuln/SNYK-RHEL7-DBUS-584292
  Introduced through: dbus@1:1.10.24-13.el7_6
  From: dbus@1:1.10.24-13.el7_6
  Fixed in: 1:1.10.24-14.el7_8


Organization:      matt-jarvis-snyk
Package manager:   rpm
Project name:      docker-image|localhost/mytest
Docker image:      localhost/mytest
Platform:          linux/amd64
Licenses:          enabled

Tested 161 dependencies for known issues, found 68 issues.

Pro tip: use `--file` option to get base image remediation advice.
Example: $ snyk test --docker localhost/mytest --file=path/to/Dockerfile

To remove this message in the future, please run `snyk config set disableSuggestions=true`

Success! Snyk is now working correctly, and talking to Podman using an unprivileged socket.

To configure DOCKER_HOST permanently in your shell, you can add the export command to your .bashrc file which will persist the setting.

Finally we’d really like to have this whole setup automatically available to us whenever we log in. To do this we can take advantage of the socket activation facilities in systemd to automatically start our socket when we log in and try to connect to it.

First we need to configure the socket :

​[matt@localhost ~]$ cat .config/systemd/user/podman.socket
[Unit]
Description=Podman API Socket
Documentation=man:podman-api(1)

[Socket]
ListenStream=/home/matt/podman.sock
SocketMode=0660

[Install]
WantedBy=sockets.target

And then tie the socket to a service :

​[matt@localhost ~]$ cat .config/systemd/user/podman.service
[Unit]
Description=Podman API Service
Requires=podman.socket
After=podman.socket
Documentation=man:podman-api(1)
StartLimitIntervalSec=0

[Service]
Type=oneshot
Environment=REGISTRIES_CONFIG_PATH=/etc/containers/registries.conf
ExecStart=/usr/bin/podman system service unix:///home/matt/podman.sock
TimeoutStopSec=30
KillMode=process

[Install]
WantedBy=multi-user.target
Also=podman.socket

Finally we reload the systemd configuration and enable the service !

[matt@localhost ~]$ systemctl --user daemon-reload
[matt@localhost ~]$ systemctl --user enable --now podman.socket

Now whenever we log in as our local user, we have the Podman API available on demand locally from an unprivileged socket, which Snyk can connect to using the DOCKER_HOST environment variable we set earlier. 

Conclusion

So there we have it, Snyk CLI image scanning with Podman working in the same way as with Docker, allowing developers easy access to security scans of local container images as part of their development workflow. 

All of the Snyk functionality is available via the CLI, including adding images to your Snyk dashboard for ongoing monitoring. Snyk collaborates with Red Hat to help developers automate security controls in application development, delivering actionable, scalable cloud native security without compromising speed or performance. Make sure your containers are scanned for security vulnerabilities, and build containers using established best practices. If you don’t already have a Snyk account, it’s free to sign up and use Snyk to scan container images and open source dependencies. 

The Docker compatibility layer in Podman provides an easy pathway to transition to using Podman without changing existing workflows and integrations. As Podman grows in popularity, it would be great to see Snyk integration with the native Podman API, but for now these compatibility functions make it super easy to integrate Podman with Snyk.