Patch to auparse to handle out of order messages 3 of 3

[Burn Alting]

On Thu, 2016-01-07 at 22:06 -0500, Paul Moore wrote:
> On January 7, 2016 6:47:02 PM Steve Grubb <sgrubb at redhat.com> wrote:
> 
> > On Friday, January 08, 2016 10:05:13 AM Burn Alting wrote:
> >> Steve,
> >>
> >> Can I suggest you modify src/ausearch-lol.c:check_events() to add in the
> >> AUDIT_PROCTITLE check (will reduce memory overhead as events will be
> >> flushed faster).
> >
> > OK. Good suggestion. The SVN repo has been updated.
> >
> >
> >> Also can we ask Richard put a comment into the appropriate location in
> >> the kernel code to indicate the link between ausearch/aurport/auparse
> >> depending on AUDIT_PROCTITLE being the last record of an event if
> >> present.
> >
> > I'll let them answer.
> 
> Good thing I happened to read this message, I had stopped reading this 
> thread...
> 
> I really dislike comment only patches and I really, really dislike the 
> fixed format fields/records/etc. that permeates so much of audit these 
> days.  I'll reserve final judgement for if/when any patches are posted, but 
> just to be clear, I'm not very excited about stuff like this.

This is just a request to the kernel audit team, to note that the user
level audit capability is making use of the AUDIT_PROCTITLE record to be
an end of event marker. If you believe this is an unacceptable risk for
downstream processing, then we can take this out and hence withdraw the
request. The alternative is to maintain status quo, and/or optionally
emit the AUDIT_EOE record into the stored audit and be done with it, and
accept the storage cost.

I wholeheartedly agree about the challenges we have with respect to the
current format of the audit events emitted by the kernel. I spend a lot
of effort converting the unstructured, sometimes inconsistently
displayed events into more structured data.

I believe the way forward is to define a more correct, efficient AND
extensible output form for the kernel. On the user space side, we assist
the existing consumers of our audit data (our customers so to speak) by
providing a legacy audispd plugin to take the 'refined' data and format
it into the current 'mash-up'. To assist this interim measure/plugin,
and state that is IS interim, when converting the kernel code to emit a
record, ensure our new method can record the old/legacy format in some
way. The legacy formatting should be able to be compiled out.

Like Paul, I don't like being 'forced' to keep bugs in place within a
data source because those downstream don't want to sustain their data
consumption capability.

> 
> > That said one of the things I want to add in the next development cycle is the
> > ability to get rid of proctitle records if the admin wants to. They waste a
> > lot of space. But if they are missing then we have the same performance as we
> > did before I added this patch.
> 
> I wouldn't have a problem with that.
> 
> >> On Thu, 2016-01-07 at 17:31 -0500, Steve Grubb wrote:
> >> > On Wednesday, January 06, 2016 09:30:36 PM Burn Alting wrote:
> >> > > #3 - modify the standard auparse() test code.
> >> >
> >> > And this patch is applied. Thanks, Burn, for all the patches! This will
> >> > make analytical programs much more accurate since interlaced records
> >> > won't split an event up any more.
> >> >
> >> > If anyone wants to try out the new audit code from svn please send any
> >> > feedback asap. (Same with other bug reports.) I am aiming for a release in
> >> > the next 2 days. I just have to finish working on Richard's audit by
> >> > process name patch and then its time to release a new package.
> 
> 
> --
> paul moore
> www.paul-moore.com
> 
>