Security monitoring in Linux with Tripwire

Every sysadmin loses sleep every once and a while over system intrusions. Nobody wants a server they're responsible for to be compromised. The problem is, even though you may review logs regularly, a truly effective system intrusion doesn't leave obvious logs lying around. This makes it difficult to know definitively whether your systems are secure.

In addition to setting SELinux to Enforcing and implementing regular pentests, one of the best ways to monitor your system for security breaches is to — well, monitor your system for security breaches. If that seems easier said than done, then you need to try Tripwire. Tripwire is a file integrity monitoring tool that watches for changes to critical files on your system. This article shows you how to install, setup, and use Tripwire on your network.

Tripwire is both a company and an open-source code base. You can purchase monitoring from Tripwire, or you can use the GPLv2 code they've made available on GitHub. The usual trade-offs apply. If you pay for it, Tripwire does most of the hard work for you, and all you have to do is pay attention to the reports. If you implement Tripwire yourself, then you get to set it up and configure it on your own.

Installing

To install Tripwire on RHEL or CentOS, you must add the Extra Packages for Enterprise Linux (EPEL) repository. On RHEL 8, you must enable the codeready-builder option in subscription-manager:

$ sudo repos --enable "codeready-builder-for-rhel-8-$(arch)-rpms"

On CentOS, you should enable PowerTools:

$ sudo dnf config-manager --set-enabled PowerTools

To install:

$ sudo dnf install -y epel-release

With EPEL now added to your repository list, install Tripwire:

$ sudo dnf install -y tripwire

Setting a hostname

Before configuring Tripwire, you should set a hostname for your server if it doesn't already have one. Hostnames are a frequent point of confusion, so read my article about setting hostnames to make sure you're clear on what you're setting. On CentOS, RHEL, and Fedora, you can set a hostname with hostnamectl:

$ sudo hostnamectl set-hostname --pretty "Rockhopper tripwire demo machine"
$ sudo hostnamectl set-hostname --static rockhopper

Generating keys

Next, you must generate encryption keys for Tripwire. After all, the point of Tripwire is to prevent attackers from covering their tracks, so Tripwire data must be strongly encrypted.

First, create a local key with the twadmin tool:

$ sudo twadmin --generate-keys --local-keyfile /etc/tripwire/$(hostname)-local.key

Next, create a site key:

$ sudo twadmin --generate-keys --site-keyfile /etc/tripwire/site.key

In both cases, you must provide a passphrase for each key. Keep these passphrases private and safe!

Tripwire uses two different keys for encryption: the local key, which is unique to each server, and a site key, which you can use across all systems within your organization. The site key is an important feature because it enables an IT manager to dictate a single security policy for the organization, and it can be updated centrally, signed with the site key, and then distributed with Ansible or scp for use on every server. Each server admin still has a unique local key, though, so even though the security policy file can't be changed, they can still access Tripwire for updates and reports.

Tripwire configuration file

Next, you need to create a basic configuration file for Tripwire. Most of the defaults in the config are acceptable, and nothing needs to be changed unless you know your system differs in some significant way from what you see in the example config file provided at /etc/tripwire/twcfg.txt. By default, Tripwire uses sendmail to email you alerts. If you're using postfix, there's no need to change it, however, because postfix provides sendmail aliases. Also defined in the config file are the locations of your encryption keys and policy file, so verify that those are correct.

When you're happy with the configuration options, use twadmin to validate the config text and write it out to a file called /etc/tripwire/tw.cfg, which is signed with the site key. Signing the configuration file requires the passphrase to your site key.

$ sudo twadmin --create-cfgfile \
--site-keyfile /etc/tripwire/site.key \
/etc/tripwire/twcfg.txt
Please enter your site passphrase:
Wrote configuration file: /etc/tripwire/tw.cfg

Policy file syntax

The policy file is where you put in most of the work for Tripwire. Your Tripwire policy dictates which files to monitor and which to ignore, and which lie somewhere in between. Both extremes are equally important. If your daily Tripwire reports send a false positive for every single user file that changes throughout a workday, then you'll quickly learn to ignore your Tripwire reports altogether.

The sample policy file bundled with the EPEL install of Tripwire is built around a full install of a Fedora Workstation. You must customize it for your system unless you're running a full install of Fedora Workstation, but reading it helps give you an idea of what a standard policy file contains. To decode the Tripwire notation, review the twpolicy(4) man page.

Policy files can be complex, and it might help to think of it more like a Sass or Makefile than a configuration file. You can create variables, or use some default ones, and rules, and even conditionals to govern how Tripwire treats individual directories and files.

For instance, the ReadOnly variable defines a rule for files meant to be read-only. In this context, "read-only" doesn't mean its file permissions are set to r-- (that is, 400), but that there's generally no change expected from the time of Tripwire's initialization to a daily report, or a report on a day-to-day basis.

A rule is structured as a line terminated by a semi-colon (;) and delimited by an arrow (->). This block of code sets the Tripwire executables to ReadOnly:

/sbin/siggen    ->   $(ReadOnly);
/sbin/tripwire  ->   $(ReadOnly);
/sbin/twadmin   ->   $(ReadOnly);
/sbin/twprint   ->   $(ReadOnly);

The sample file from Fedora uses variables to define most rules. For example:

SEC_BIN = $(ReadOnly) ;  # Binaries that should not change

So its entries to monitor Tripwire binaries are:

/sbin/siggen    ->   $(SEC_BIN);
/sbin/tripwire  ->   $(SEC_BIN);
/sbin/twadmin   ->   $(SEC_BIN);
/sbin/twprint   ->   $(SEC_BIN);

The two examples serve exactly the same purpose, and the implementation is just different.

Use the sample policy file as a starting point and construct a policy to monitor your system.

Generating a policy file

Exclusively for the sake of testing, add this line to the Tripwire Data Files section:

/etc/tripwire/secrets -> $(SEC_CRIT); # proof of concept

Create a critical test file named secrets:

$ sudo touch /etc/tripwire/secrets

Generate your policy file with twadmin:

$ sudo twadmin --create-polfile \
--site-keyfile /etc/tripwire/site.key \
/etc/tripwire/twpol.txt

Enter your site key passphrase when prompted.

This results in a signed and encrypted file, /etc/tripwire/tw.pol.

Initializing Tripwire

With your keys generated, your configuration set, and a policy file in place, you can now initialize Tripwire:

$ sudo tripwire --init

Enter your local key passphrase when prompted.

If you see warnings, read them carefully and correct the errant entries in your policy file. It's not uncommon for your first attempt at a policy file, especially when it's based on an existing one, to reference files that don't actually exist on your system. You can deal with this by installing the missing files or by removing the references to them from your /etc/tripwire/twpol.txt master file.

If you had to make changes, update your policy file by regenerating it, and then re-initialize your database:

$ sudo twadmin --create-polfile \
--site-keyfile /etc/tripwire/site.key \
/etc/tripwire/twpol.txt
$ sudo tripwire --init

You should do this until you have reached a good starting place. Once you've got a sane starting database, you shouldn't re-initialize your database, but instead use the tripwire command to check the integrity of your system and, optionally, override acceptable differences with the --interactive option:

$ sudo tripwire --check --interactive

Viewing reports

The EPEL install of Tripwire creates cron jobs to run Tripwire reports, and to email reports to root. You can run a manual report, too:

$ sudo tripwire --check

This command saves a report file to /var/lib/tripwire/reports (or whatever location you set in the configuration file). To view this file, use the twprint command:

$ sudo twprint --print-report --report-level 1 \
--twrfile /var/lib/tripwire/reports/`hostname`-20200317-163425.twr

To see a report with an error, make a change the secrets test file and run a report:

$ sudo echo 1 > /etc/tripwire/secrets
$ sudo tripwire --check

Then view the report:

$ sudo twprint --print-report --report-level 1 \
--twrfile /var/lib/tripwire/reports/`hostname`-20200317-164413.twr
Added: "/var/lib/tripwire/rockhopper.twd.bak
Modified: "/etc/tripwire/secrets"

Assuming you're happy with the modification to your test file, you can update Tripwire's database:

$ sudo tripwire --update \
--twrfile /var/lib/tripwire/reports/`hostname`-20200317-164413.twr

Protect your systems

Tripwire is a highly-precise and extremely pedantic security monitor. Stop struggling to parse logs for signs of intruders and make Tripwire work for you. With Tripwire, when something changes on a system, you'll know about it, and you can deal with it accordingly.

[ Want to learn more about security? Check out the IT security and compliance checklist.  ]