More tips for packaging your Linux software with RPM

If you use Red Hat Enterprise Linux (RHEL) or Fedora, you will soon find yourself using dnf (or yum) to install software packages. Red Hat Package Manager (RPM) is the most important software management tool on these Linux distributions. This article shows how you can take advantage of this framework to distribute your own applications.

Hopefully, you've had a chance to read Valentin Bajrami's article on the same subject. I will repeat some of the same concepts here and illustrate a few issues you may find along the way. I will also use two more complex examples and identify a few problems you may discover when packaging native applications.

This article shows you how to determine how to compile and package your own native application for distribution. In a follow-up article, I'll explain how to package third-party applications that do not have an RPM package, or if they do, you want to customize it.

Prerequisites include:

• You have basic knowledge of how to use RPM to query packages and install or delete packages. If not, get familiar with these concepts first and then come back here for some fun.
• You have Make, Git, GCC, and Java installed, as you'll need them to complete the exercises included here. It's not required, but it would be nice if you practice as I move along.

To install Make, GCC, Java 11, and Git using the DNF package manager, run:

$ sudo dnf install \
make gcc-10 java-11-openjdk-headless git

Package your own software using RPM

This article uses a small open source project called jdumpertools for this step.

On your Linux terminal, clone jdumpertools and then compile it (you have installed Make and the GCC compiler, right?):

$ git clone git@github.com:josevnz/jdumpertools.git
$ cd jdumpertools
$ make all

$ ls -l jdu jutmp *.so
-rwxrwxr-x 1 josevnz josevnz 32728 Oct  3 16:40 jdu
-rwxrwxr-x 1 josevnz josevnz 32752 Oct  3 16:40 jutmp
-rwxrwxr-x 1 josevnz josevnz 29024 Oct  3 16:40 libjdumpertools.so

Then you can run any of the generated programs. For example, try out jdu (a simpler version of the du command that prints the results in JSON format):

$ LD_LIBRARY_PATH=$PWD $PWD/jdu /
[{"partition": "/", "free_space": 462140129280.000000, "total_space": 510405902336.000000}]

So far, so good.

The jdumpertools.spec file in this directory is the RPM specification file that controls how to compile and package jdumpertools using RPM.

Next, install a few supporting tools before moving to building the RPM file.

Install RPM building blocks

Get the necessary rpmdevtools utilities by running:

$ sudo dnf install rpm-build rpmdevtools

Then prepare the sandbox to build RPMs using rpmdevtools. Never use root for this step but rather your personal or developer Linux account (it will pass the -d debug flag):

$ rpmdev-setuptree -d
josevnz       /home/josevnz    /home/josevnz/.rpmmacros
/home/josevnz/rpmbuild/RPMS    /home/josevnz/rpmbuild/SOURCES     /home/josevnz/rpmbuild/SPECS
/home/josevnz/rpmbuild/SRPMS   /home/josevnz/rpmbuild/BUILD

Here's a nicer view:

$ tree ~/rpmbuild
/home/josevnz/rpmbuild
├── BUILD
├── RPMS
├── SOURCES
├── SPECS
└── SRPMS

5 directories, 0 files

Right now, you need to pay attention to only two directories: SOURCES and SPECS. I will explain the others later.

There's also a new file called ~/.rpmmacros. You can put or override some special macros there to avoid repetitive tasks while building RPM packages. This file is important because it anchors the rpmbuild environment with your home directory.

Package your application

First, create a tar file of the source code in the ~/rpmbuild/SOURCES directory:

VERSION=v0.1
NAME=jdumpertools
TARFILE=$(NAME)-$(VERSION).tar.gz
/usr/bin/tar --exclude-vcs --directory ../ --create --verbose --gzip --file $(HOME)/rpmbuild/SOURCES/$(TARFILE) $(NAME)

Normally the tar file contains scripts and source code you will compile within the packaging process.

Next, create an RPM spec file. Again, rpmdevtools provides a head start, like this:

$ rpmdev-newspec ~/rpmbuild/jose-package.spec
/home/josevnz/rpmbuild/jose-package.spec created; type minimal, rpm version >= 4.16.

$ cat  ~/rpmbuild/jose-package.spec
Name:           jose-package
Version:        
Release:        1%{?dist}
Summary:        

License:        
URL:            
Source0:        

BuildRequires:  
Requires:       

%description

%prep
%autosetup

%build
%configure
%make_build

%install
rm -rf $RPM_BUILD_ROOT
%make_install


%files
%license add-license-file-here
%doc add-docs-here

%changelog
* Sun Oct 03 2021 Jose Vicente Nunez <josevnz@kodegeek.com>
- 

Don't worry if you can't make sense of this file now. Copy the jdumpertools.spec file to the ~/rpmbuild/SPECS directory:

$ cp -pv jdumpertools.spec ~/rpmbuild/SPECS

And create a source and a binary RPM file:

$ rpmbuild -ba rpmbuild/SPECS/jdumpertools.spec
setting SOURCE_DATE_EPOCH=1609718400
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.kBlIwO
+ umask 022
+ cd /home/josevnz/rpmbuild/BUILD
+ cd /home/josevnz/rpmbuild/BUILD
+ rm -rf jdumpertools
+ /usr/bin/gzip -dc /home/josevnz/rpmbuild/SOURCES/jdumpertools-v0.1.tar.gz
+ /usr/bin/tar -xof -
+ STATUS=0
+ '[' 0 -ne 0 ']'
+ cd jdumpertools
+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ RPM_EC=0
[...]
Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/josevnz/rpmbuild/BUILDROOT/jdumpertools-v0.1-1.fc33.x86_64
Wrote: /home/josevnz/rpmbuild/SRPMS/jdumpertools-v0.1-1.fc33.src.rpm
Wrote: /home/josevnz/rpmbuild/RPMS/x86_64/jdumpertools-v0.1-1.fc33.x86_64.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.uEyCyL
+ umask 022
+ cd /home/josevnz/rpmbuild/BUILD
+ cd jdumpertools
+ rm -rf /home/josevnz/rpmbuild/BUILDROOT/jdumpertools-v0.1-1.fc33.x86_64
+ RPM_EC=0
++ jobs -p
+ exit 0

The end result is two files: a source RPM and a binary RPM.

Wrote: /home/josevnz/rpmbuild/SRPMS/jdumpertools-v0.1-1.fc33.src.rpm
Wrote: /home/josevnz/rpmbuild/RPMS/x86_64/jdumpertools-v0.1-1.fc33.x86_64.rpm

What happens during an RPM install

So what happens when you install each of your custom RPMs?

1. Installing the source RPM creates the tar file and the spec file in your rpmbuild directory. This allows you to recompile the application, make fixes to the RPM spec file, etc.

   $ ls rpmbuild/{SPECS,SOURCES}
   rpmbuild/SOURCES:
   jdumpertools-v0.1.tar.gz
   
   rpmbuild/SPECS:
   jdumpertools.spec

2. If you install the binary RPM, then you are actually installing the application:

   $ sudo rpm -ihv ~/rpmbuild/RPMS/x86_64/jdumpertools-v0.1-1.fc33.x86_64.rpm
   Verifying...                ################ [100%]
   Preparing...                ################ [100%]
   Updating / installing...
      1:jdumpertools-v0.1-1.fc33  ################ [100%]

3. Confirm the installed package is present:

   $ rpm -qi jdumpertools
   Name        : jdumpertools
   Version     : v0.1
   Release     : 1.fc33
   Architecture: x86_64
   Install Date: Sun 03 Oct 2021 06:32:50 PM EDT
   Group       : Unspecified
   Size        : 95002
   License     : Apache License 2.0
   Signature   : (none)
   Source RPM  : jdumpertools-v0.1-1.fc33.src.rpm
   Build Date  : Sun 03 Oct 2021 06:27:11 PM EDT
   Build Host  : dmaf5.home
   URL         : https://github.com/josevnz/jdumpertools
   Summary     : Programs that can be used to dump Linux usage data in JSON format.
   Description :
   
   Jdumpertools is a collection of programs used to dump Linux usage data in JSON format to be ingested by other tools.
   
   * jdu: Similar to UNIX 'du' command.
   * jutmp: UTMP database dumper

Note: Curious readers probably opened the Makefile and saw a target called rpm:

rpm: all
    test -x /usr/bin/rpmdev-setuptree && /usr/bin/rpmdev-setuptree|| /bin/mkdir -p -v ${HOME}/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
    /usr/bin/tar --exclude-vcs --directory ../ --create --verbose --gzip --file $(HOME)/rpmbuild/SOURCES/$(TARFILE) $(NAME)
    /usr/bin/rpmbuild -ba jdumpertools.spec

This is a convenient shortcut to prepare an rpmbuild sandbox, tar files after they are compiled with make, and package them using the rpmbuild command. Feel free to call make rpm and see what happens.

[ Get more tips by downloading the Bash shell scripting cheat sheet. ]

I showed you the commands and tools used to generate the RPM files, but now it is time to write the RPM spec file.

Create a spec file for jdumper tools

Working with spec files requires filling several metadata tags as well as describing how you will compile or package the application. Jdumpertools is a simple ANSI C application, so unpack the sources, compile them, copy them into an intermediate area (~/rpmbuild/BUILDROOT), and then package them for distribution.

First, take a look at the RPM spec file:

Name:           jdumpertools
# TODO: Figure out a better way to update version here and on Makefile
Version:        v0.1        
Release:        1%{?dist}
Summary:        Programs that can be used to dump Linux usage data in JSON format. 

License:        Apache License 2.0
URL:            https://github.com/josevnz/jdumpertools
Source0:        %{name}-%{version}.tar.gz

BuildRequires:  bash,tar,gzip,rpmdevtools,rpmlint,make,gcc >= 10.2.1
Requires:       bash

%global debug_package %{nil}

%description

Jdumpertools is a collection of programs you can use to dump Linux usage data in JSON format to be ingested by other tools.

* jdu: Similar to UNIX 'du' command.
* jutmp: UTMP database dumper

%prep
%setup -q -n jdumpertools

%build
make all

%install
rm -rf %{buildroot}
/usr/bin/mkdir -p %{buildroot}/%{_bindir}
/usr/bin/cp -v -p jdu jutmp %{buildroot}/%{_bindir}
/usr/bin/mkdir -p %{buildroot}/%{_libdir}
/usr/bin/cp -v -p libjdumpertools.so %{buildroot}/%{_libdir}

%clean
rm -rf %{buildroot}

%files
%{_bindir}/jdu
%{_bindir}/jutmp
%{_libdir}/libjdumpertools.so
%license LICENSE
%doc README.md


%changelog
* Mon Jan  4 2021 Jose Vicente Nunez <kodegeek.com@protonmail.com> - 0.1
- First version being packaged

Review what is important here:

• The metadata, including version, name, and Source0; you can use variables or macros
• Unpack the sources in the %prep section using the %setup macro (the RPM guide has plenty of details about the flags)
• BuildRequires: You must list the dependencies you need to build the package. These cannot be dynamically detected.
• %build section: Compile with make
• %install section: Copy what you need for the program to work (program, libraries, etc.)
• %files section: Where you can specify if a file is a document (%doc), license file (% license), or a regular file

Also important:

• I disabled the creation of debug code during packaging with %global debug_package %{nil}.
• The changelog documents what changed with this new package version.

Check for errors in the spec file with rpmlint

You don't want to find out the hard way that your RPM is not perfect. Therefore, it is good to check for obvious errors or ways to improve your RPMs:

$ sudo dnf install rpmlint

Now check the RPM spec file:

$ rpmlint /home/josevnz/rpmbuild/SPECS/jdumpertools.spec 
/home/josevnz/rpmbuild/SPECS/jdumpertools.spec: W: invalid-url Source0: jdumpertools-v0.1.tar.gz
0 packages and 1 specfiles checked; 0 errors, 1 warnings.

The rpmlint documentation says that Source0 must be a well-defined URL (the value should be a valid public HTTP, HTTPS, or FTP URL). Do not worry about this warning.

What about the RPM itself?

$ make rpm
...
$ rpmlint --info ~/rpmbuild/RPMS/x86_64/jdumpertools-v0.2-1.fc33.x86_64.rpm
jdumpertools.x86_64: W: summary-ended-with-dot C Programs that can be used to dump Linux usage data in JSON format.
jdumpertools.x86_64: W: spelling-error %description -l en_US du -> dew, doe, Du
jdumpertools.x86_64: E: description-line-too-long C Jdumpertools is a collection of programs that can be used to dump linux usage data in JSON format, so it can be ingested by other tools.
jdumpertools.x86_64: W: incoherent-version-in-changelog 0.1 ['v0.1-1.fc33', 'v0.1-1']
jdumpertools.x86_64: W: invalid-license Apache License 2.0
jdumpertools.x86_64: W: unstripped-binary-or-object /usr/bin/jdu
jdumpertools.x86_64: W: unstripped-binary-or-object /usr/bin/jutmp
jdumpertools.x86_64: W: unstripped-binary-or-object /usr/lib64/libjdumpertools.so
jdumpertools.x86_64: W: no-soname /usr/lib64/libjdumpertools.so
jdumpertools.x86_64: W: no-manual-page-for-binary jdu
jdumpertools.x86_64: W: no-manual-page-for-binary jutmp
1 packages and 0 specfiles checked; 1 errors, 10 warnings.

Ten warnings and one error. Some are easy to fix:

• The license must be a specific format
• Man pages are required for programs, so I wrote a very simple one with troff
• Include the soname in the library

After the fixes, only one warning remains:

$ make rpm
...
$ rpmlint --info ~/rpmbuild/RPMS/x86_64/jdumpertools-v0.2-1.fc33.x86_64.rpm 
jdumpertools.x86_64: W: spelling-error %description -l en_US du -> dew, doe, Du
The value of this tag appears to be misspelled. Please double-check.

This warning is alerting you that the word du appears to be misspelled. It's just a reference to a valid command, though, so you can ignore the warning.

Now upgrade the RPM with the improved version:

$ sudo rpm -Uhv ~/rpmbuild/RPMS/x86_64/jdumpertools-v0.2-1.fc33.x86_64.rpm

I use the rpm command to make it more obvious that I have upgraded the package from a local disk version instead of a new version from a repository. If you prefer, you can do the same thing with dnf:

$ sudo dnf install --upgrade \
~/rpmbuild/RPMS/x86_64/jdumpertools-v0.2-1.fc33.x86_64.rpm

What you learned, and what is next

Packaging software with RPM looks intimidating at first, but with a little bit of patience, you will get there in no time. As you encounter issues, you will also find proper ways to improve your code. Below are some resources and final recommendations:

• Do yourself a big favor and get a copy of the RPM Packaging Guide written by Adam Miller, Maxim Svistunov, and Marie Doleželová. It is very complete and well organized. Seriously, do it now; it is that good.
• The official RPM Packaging Guide and the Fedora RPM guide are also full of details; keep them a bookmark away.
• Use rpmlint. You will be surprised how many little things you can catch and fix before shipping your RPM packages.
• Not enough? Fedora has a list of tricks you can use when packaging software.
• Still curious? You should definitely take a look at RPM Packaging guidelines.