New draft standards

[Burn Alting]

On Tue, 2015-12-08 at 19:28 -0500, Paul Moore wrote:
> On Tuesday, December 08, 2015 03:25:22 PM Steve Grubb wrote:
> > On Tuesday, December 08, 2015 02:58:18 PM Paul Moore wrote:
> > > On Tue, Dec 8, 2015 at 2:22 PM, Steve Grubb <sgrubb at redhat.com> wrote:
> > > > Hello,
> > > > 
> > > > I would like to point out 2 new standards that have been posted to the
> > > > linux audit web page. The first establishes the events around system
> > > > start up and shutdown. This is important because it sets the session
> > > > boundaries for when a system is up or down or crashed.
> > > > 
> > > > http://people.redhat.com/sgrubb/audit/system-lifecycle.txt
> > > > 
> > > > The second standard is more of a forward looking standard. It explains
> > > > how
> > > > the audit daemon and utilities will perform event enrichment before
> > > > being
> > > > stored long term in an aggregator. The target for implementation is the
> > > > 2.5 release of the audit daemon.
> > > > 
> > > > http://people.redhat.com/sgrubb/audit/event-enrichment
> > > > 
> > > > Let me know if anyone has feedback on these standards, especially the
> > > > second one.
> > > 
> > > Were these two specification documents created based on published
> > > standards from an established standards body, e.g. NIST, IETF, etc?
> > 
> > No. All of the standards published to date is documenting what exists and
> > why. The needs are typically driven by common criteria and the need to
> > detect certain kinds of events for intrusion detection or anomalous
> > conditions.
> 
> Okay, let's not call these "standards" and just stick with "specifications".  
> The term standards has all sorts of connotations associated with it, both good 
> and bad, and I think we should be clear when we start talking with other 
> developers.  I think it would also be *very* helpful if you could explain the 
> motivation behind these specs so we understand what problems you are trying to 
> solve and what requirements you are trying to meet; you talk about this a bit 
> in the conclusion, but more background would be nice.
> 
> Another nit-picky comment, in the future I would suggest sending the specs 
> inline in your mail; having to go to my browser to read your document and then 
> cut-and-paste it into my email to comment on it means your request for 
> feedback goes to the bottom of my todo list.  Lower the bar for collaboration 
> as much as possible, if you inline the text all we have to do is hit "reply" 
> to comment on the specs.
> 
> Anyway, on to your docs ...
> 
> * https://people.redhat.com/sgrubb/audit/system-lifecycle.txt
> 
> > System Lifecycle Auditing
> > =========================
> > This document will go over the events that are associated with starting up
> > a system and shutting it down. Knowing what events make up these actions
> > allows an analytical application to know the boundaries of all sessions and
> > actions a user may perform. It also allows identification of crashed systems
> > or malfunctioning services. The following table lists the events that make
> > up the system lifecycle in the audit trail:
> > 
> > AUDIT_SYSTEM_BOOT - System boot
> > AUDIT_SYSTEM_RUNLEVEL - System runlevel change
> > AUDIT_DAEMON_START - Audit daemon startup record
> > AUDIT_DAEMON_ABORT - Audit daemon error stop record
> > AUDIT_SERVICE_START - Service (daemon) start
> > AUDIT_SERVICE_STOP - Service (daemon) stop
> > AUDIT_SYSTEM_SHUTDOWN - System shutdown
> > AUDIT_DAEMON_END - Audit daemon normal stop record
> 
> Why both an AUDIT_DAEMON_ABORT and an AUDIT_DAEMON_END and not just an 
> AUDIT_DAEMON_STOP with a field to indicate if it was a normal shutdown or a 
> failure as outlined in the spec?  This would be more consistent with the other 
> daemons and the shutdown result field could potentially be reused by the init 
> systems for other daemons (assuming the information was conveyed via return 
> values or some other mechanism).
> 
> > Lifecycle of the system
> > =======================
> > When the system is powered on, control is eventually turned over to an
> > init daemon. This daemon is responsible for starting up all other system
> > services and performing an order system shutdown when asked. This init
> > daemon should send a AUDIT_SYSTEM_BOOT event after it has done its own
> > initialization. Most init systems have different targets or modes of
> > operation that the system is turned over for interactive sessions. Examples
> > are multi-user console, multi-user graphical, etc. 
> 
> You mention it later, but it might be a good idea to mention in this paragraph 
> that the audit daemon should be started as early as possible by the init 
> daemon.
> 
> > Init will determine what runlevel the system is ultimately going to try to
> > achieve. When gets there or it fails to get there, it shall issue an
> > AUDIT_SYSTEM_RUNLEVEL event to denote which mode of operation it was going
> > to be in. If an admin requests that the system switch to another runlevel,
> > then it should issue another AUDIT_SYSTEM_RUNLEVEL event.
> 
> I think it would be good to have a discussion about runlevels that don't 
> follow the traditional integer numbering, e.g. string based runlevels.
> 
> * https://people.redhat.com/sgrubb/audit/event-enrichment
> 
> > Audit Event Enrichment
> > ======================
> > 
> > There are times when the audit events are stored in another machine and need
> > to be searched at a later date. Some parts of the audit event are temporally
> > limited in duration or unique to a system. This makes interpretting fields
> > that are numbers into human readable fields hard or impossible without
> > running a report at the time of the event or on the machine the event
> > occurred on.
> > 
> > To address this issue, the audit daemon will get a new mode, enrich, where
> > the audit trail will be amended as follows at the time an event is logged:
> > 
> > 1) Translations will be:
> >   a) appended to the end of the event with the field's name in capital
> >      letters
> 
> Please no, let's leave field names case insensitive, perhaps an agreed upon 
> suffix, e.g. "-trans"?
> 
> >   b) encoded if user controlled data is used for enrichment
> > 
> > 2) The auparse library will:
> >   a) preferentially use these fields whenever an interpretation is requested
> >   b) if none exist, look up the fields on the local machine if necessary
> 
> I think resolving these fields on the local machine is misleading, and 
> potentially dangerous; this is especially true with respect to SELinux labels.  
> If you can't resolve the field, simply display it raw and let the operator 
> determine how to handle it.
> 

Steve,

Can you mock up some examples of an 'enriched' event showing how it is
different from what we have now.

Being one of those people who maintain a central repository of Linux
audit, my main "ingest" concerns are to have data that is simple and,
hence, quick to parse and hence normalize.

I think the risks associated with resolution of raw data can be
mitigated by optionally maintaining the raw value as well when
transmitting the event to a central repository.

Given the above is implemented, then I would recommend the modification
of ausearch to optionally translate a complete raw enriched event into
either a single json or xml record and optionally include raw and
interpreted values. The decision to include both could be driven via
pattern matched directive (eg *id|hostname). In reality, irrespective of
whether the above is implemented or not, I would recommend (and will
probably create the patch). 

To me the biggest benefit of Steve's proposal is the near real time
resolution of some values. This is particularly useful for IP addresses
(given one also notes somewhere, the name servers the system uses for
resolution) given their potential reuse in short periods of time.

Regards
Burn