Red Hat Device Edge RPM-OSTree updates

Connectivity at the edge is not like the datacenter where there is plenty of bandwidth and low latency via the interconnect of the systems. Devices at the edge are often hampered by low bandwidth, high latency, intermittent connectivity or a combination of all three. Red Hat Device Edge helps overcome many of these challenges using a unique system called  RPM-OSTree.

RPM-OSTree is, by definition, a hybrid image/package system. An image/package system bundles all of the components needed for our system configuration into a single commit which can be deployed across multiple devices. But that is only half the story as after the initial deployment we still need to apply a commit to the device. Which is why, in most cases, the initial commit is laid down before the device ships to the edge site.

Traditional RPM systems use repositories of packages and assemble the install locally on a system. The build process for an RPM-OSTree system works similarly using those same repositories. What’s different is the package transaction is stored in the commit similar to how Git works. We can then make this available over a web server in a similar manner to an RPM or Git repository. With this, systems deployed in the field can pull new commits, such as OS updates, as they become available. 

RPM-OSTree has another trick up its sleeve when it comes to the Day Two update operations once the device is in the field. Similar to dnf deltarpm updates, RPM-OSTree updates can be done based on differential commits or, in other words, only the parts of the package that have truly changed. Not only do we avoid downloading RPM metadata, we can also further optimize these updates by computing a static-delta which will compress and lower the TCP connection overhead of systems' pull updates. You could think of it almost like a snapshot seen on many storage subsystems.  However unlike dnf deltarpm updates, which come with a CPU cost on the edge device, RPM-Ostree differential commits do not have such costs.

What’s key to understand is that an RPM-OSTree system will stage these updates in the background and the current state of the running system is not changed. This greatly decreases the risk of snowflake systems in a large fleet of devices. The cost of immutable updates is that a reboot is required. This is ultimately a small price to pay given the many benefits such as transactional updates, staging updates, delta updates, rollback on failure and immutable OS.

To understand the RPM-OSTree updates concept let's take a closer look at an example. If we look at the example deployment below we have a collection of system packages that you might see at the edge: kernel, podman, cri-o, libvirtd, openssl and MicroShift. Together these make up an initial deployment that we might apply across some edge devices using either the traditional RPMs or RPM-OSTree image methods.   

In our example deployment we currently have 6 packages, with 20 parts per package. Again, these values are arbitrary and don’t represent the exact files of the packages—these are being used to demonstrate the concept here.

Now that we have laid down the deployment on our devices, in the future we might need to patch for security vulnerabilities or additional feature functionality.  Let’s take a look at what that patching process looks like from a traditional RPM package deployment scenario (ie non-RPM-OSTree).  We determined that the following packages needed to be patched: kernel, cri-o, openssl and MicroShift.  With the traditional RPM package model we have to download the full package for each of those updates even if some of the code inside those packages has not changed. We can see the total amount we would need to download is 80 parts..

80 files is still a lot of data to move to the edge. In fact it’s 66% of the total deployment size!  Now let’s take a look at the RPM-OSTree differential commit update for the same packages. In this scenario, RPM-OSTree will create an update image that only contains the updated commits, reducing the amount of data that needs to be transmitted to the edge device. When using RPM-OSTree, the transmission size drops to only 28 delta parts; that's 52 fewer parts than with RPMs!

The considerable reduction in deployment size when using image-based deployments such as RPM-OSTree provides a significant advantage over traditional RPM-based methods. In edge environments, with constrained connectivity resources, this savings is essential and critical to success.

Learn more

• Meet Red Hat Device Edge with MicroShift
• Red Hat Device Edge official documentation
• Creating Red Hat Device Edge images