How to generate a sosreport for Red Hat Enterprise Linux CoreOS

With the introduction of Red Hat OpenShift Container Platform 4 in June 2019, Red Hat also introduced Red Hat Enterprise Linux (RHEL) CoreOS. RHEL CoreOS represents the next generation of single-purpose container operating system technology. RHEL CoreOS was created by the same development teams that created Red Hat Enterprise Linux Atomic Host and CoreOS Container Linux, and combines the quality standards of Red Hat Enterprise Linux (RHEL) with the automated, remote upgrade features from Container Linux. Additionally, RHEL CoreOS is the new underlying operating system required for the master nodes along with the default operating system for the worker nodes.

While the underlying operating system consists primarily of RHEL components and the same quality, security, and control measures that support RHEL, there is one key feature that is not included in RHCOS: the sos package.

With the exclusion of the sos package, which provides the sosreport application, naturally we ask ourselves, “How do we generate a sosreport in RHCOS?” Thankfully, Red Hat provides two options to generate a sosreport from a RHCOS host. The first, which is the preferred method, is to debug the node using the oc debug command and the second is to connect to the host via SSH. 

Connecting to a RHEL CoreOS 4.x node without SSH access using oc debug

To begin you will need to list all of your nodes and then select the node you wish to debug.

In this example, we will debug master node master1.openshift.lab.int by running oc debug node/master1.openshift.lab.int. The cluster will spawn a debug pod using image registry.redhat.io/rhel7/support-tools. Once inside the debug pod you will then chroot into the underlying host by running chroot /host bash.

Connecting to a RHEL CoreOS 4.x node using SSH

As a reminder, while connecting to the underlying Red Hat OpenShift nodes via SSH to generate a sosreport is possible, it is not recommended. The recommended process is to debug the node using the ‘oc debug’ command.

By default, the Red Hat OpenShift 4 installer creates a single user named core with the optional SSH keys provided at the time of installation. In the example below a custom key is specified from the default SSH key location.

Using the toolbox script to run the sosreport

Toolbox is a small script that launches a container that lets you bring in your favorite debugging or admin tools. For RHEL CoreOS this will start a container using the registry.redhat.io/rhel8/support-tools:latest image.

After the toolbox container successfully launches, you will have the ability to run the sosreport application to generate the sosreport for your support ticket.

Once your sosreport has been generated the file will be saved in /var/tmp on the RHEL CoreOS node. If your nodes have access to the internet you can upload the sosreport directly to your case using the redhat-support-tool application which is also included in the toolbox container.

Summary

In this post, we looked at the two different ways to access a RHEL CoreOS node using the recommended method of running the oc debug command and by connecting via SSH. Additionally, we covered running the toolbox script which creates a container that contains the binary for generating a sosreport from a Red Hat Enterprise Linux CoreOS. Finally, we provide an example of how to use the redhat-support-tool to upload your sosreport directly to your case if your node has the proper internet access required.