Audit for live supervision

[Steve Grubb]

On Tuesday 19 August 2008 02:45:00 Kay Hayen wrote:
> Hello Steve,
>
> > > I tried to change our rules to "exit,always" from "entry,always", but
> > > it didn't make a difference. Can you confirm that only one exit is
> > > traced and do you think audit can be enhanced to trace these extra
> > > exits of syscalls like FORK.
> >
> > Yes, I think the kernel could be updated to return twice. This would need
> > to be sent upstream and I think 2.6.28 is the next chance.

<snip>

> Is there any hope such a patch could be part of RHEL 5.3, given that Redhat
> has its own kernel release process? I am not that much into security, but I
> could imagine that it's possible to carefully craft a process that escapes
> the audit trail with SIGKILL to a forker.

Just because a process exists is not a security concern. The process actually 
has to do something related to security - e.g. access resources. At that 
point we will pick it up. I can see that we should probably have 2 records on 
the clone syscall if that is being audited.


> All you got to do is to fork a process that will fork another and with
> increasingly bigger times, you SIGKILL the process until its child will
> secretely survive.

Sure, but as soon as it touches something or makes any syscall, its 
potentially auditable.


> > > > ausyscall x86_64 clone
> > > > 56
> > > >
> > > > ausyscall i386 clone
> > > > 120
> > >
> > > Very good. We have initially defined a hash in Python manually with
> > > what we encounter, but we can rather use that to create them. We
> > > specifically have the problem of visiting a s390 site, where it will
> > > handy to have these already in place. There is no such function in
> > > libaudit, is there?
> >
> > For what?
>
> Well for the functionality of ausyscall. If we could query the current
> arch, well it's b32/b64 arches, then we could build that table at run time,
> couldn't we?

Sure. If you look at the code for ausyscall, it simply calls 
audit_syscall_to_name() in libaudit. On number to string its a straight 
lookup, for string to number, we have to brute force search for it. 


> That would be a whole lot nicer than hardcoded values, even if they are
> generated using ausyscall.

Sure. Occassionally syscalls get added to the upstream kernel and very rarely 
to a RHEL kernel. So, using libaudit would future-proof the code.


> > > > We have an audit parsing library. It takes this into account.
> > >
> > > I have looked at it, and auparse_init doesn't seem to support reading
> > > from the socket itself, does it?
> >
> > You mean the netlink socket?
>
> No, when opening the socket the to the sub deamon audisp. I couldn't
> convice myself how the API would work with a socket. Does it?

Not directly because the audit internal API has the type as an integer 
separate from the text of the event. Its really simple to create a string 
that auparse can use and then use the feed interface. A working example of 
the feed interface can be found in audisp/plugins/prelude/audisp-prelude.c.


> > > In an ideal world, we would like to note that the audit socket is
> > > readable, hand it (or an arbitrarily truncated chunk of data) from it
> > > to libaudit, ask it for events until it says there are not more. That
> > > would leave the truncated line/event issue to libaudit. Is that part of
> > > the code?
> >
> > libaudit should pull complete events from the kernel unless an execve has
> > an excessive number of arguments or large sized arguments.
>
> I read that as that we can use the netlink socket with the libaudit
> directly, which sort of could be exactly what we want. That would mean we
> wouldn't use audit user space (processes) at all, right?

True. You would have to load your own rules since that is done by the audit 
user space.


> > > Note that we get a SYSCALL with 2 items, and then in order the items -
> > > from the socket. But inbetween we get type=EXECVE it doesn't have an
> > > item number,
> >
> > I suppose that could be fixed.
> >
> > > and worse the new line before 'a1=--color-auto' is real and so is
> > > the empty line after it. I have another example of a "gnash" call from
> > > Konqueror with no less than 29 arguments.
> >
> > That is coming from here, and I think a patch was submitted fixing it.
> >
> > http://lxr.linux.no/linux+v2.6.26.2/kernel/auditsc.c#L1114
>
> I see. Strange to see line formatting like that in the kernel in the first
> place. But libaudit doesn't care about them anyway I suppose.

No it doesn't. Things down stream from it might, but its stripped going to 
disk.


> > > > > I have no idea how much it represents and existing external
> > > > > interface, but I can imagine you can't change it (easily). Probably
> > > > > the end of type= must be detected by terminating empty line in case
> > > > > of those that can be continued. But it would be very ugly to have
> > > > > to know the event types that have this so early in the decoding
> > > > > process.
> > > >
> > > > We have a parsing library, auparse, that handles the rules of audit
> > > > parsing. Look for auparse.h for the API.
> > >
> > > If you confirm that can handles the parsing from the socket, as
> > > suggested above, we may persue that path and can ignore strangeness of
> > > the format once its handled by the library.
> >
> > The audit parsing library wants to read text strings as you would find
> > them on disk. The kernel keeps type separate as an integer so that
> > decisions can be made about what the record means without having to do a
> > text to int conversion. So, the audit daemon does the reformatting after
> > it decides that it a record type that we are interested in.
>
> And I read that as the libaudit library being unable to use the netlink
> socket directly.

No, libaudit is the I/O inteface with the kernel. The audit daemon and 
auditctl make extensive use of it when talking to the kernel about audit 
events. wrt auparse (if that's what you meant) you just run the data through:

asprintf(&v, "type=%s msg=%.*s\n", type, e->hdr.size, e->data);

and "v" has the string ready for auparse use. asprinf() allocates memory, so 
watch that it doesn't create a memory leak.


> [ Options for listening]
>
> > You have 4 points to get the audit stream, in order of distance from the
> > event generation: the audit netlink socket, auditd realtime interface,
> > audisp plugin interface, and the af_unix socket created by the af_unix
> > plugin from audispd. For higher reliability where you don't want of need
> > any other audit processing interfering, I would say use either of the
> > first 2.
>
> The latency is getting higher with each step. For optimal performance we
> would listen to the netlink socket and duplicate only the code essential to
> process what we are interested it.

Sure


> For extra points and hurt, we would do it in Ada and inside the target
> process, really achieving the low latency. It may be the only realistic
> option, but it also feels like duplication of effort. We have done netlink
> interfaces in Ada before, but also have on our mind that it was said that
> the netlink interface was said (not by you) to be still in flux. Is that
> still true?

We are in the process of migrating from the old rules to the new rules API. 
from kernel 2.6.6 to around 2.6.16 had one API (audit_add_rule) and replaced 
with a new and improved API (audit_add_rule_data) for kernels after that. The 
deprecated functions should be removed from libaudit.h so that there is 
binary compatibility for prebuilt apps and newly built apps won't be able to 
use the old functions.


> It certainly would be nice if the audisp had some form of output that can
> be fed directly into libaudit parsing as it comes in. But that may be an
> unrealistic expectation, is it?

one note...libaudit is an I/O library, libauparse is the library that parses 
audit events. Assuming you meant the latter...they are built for one another. 
The audispd feeds data to siblings. In the configuration file, you just 
specify that the child app wants string data and it takes care of  the 
conversion. The prelude plugin is a good example. However, audispd plugins 
are probably too high of latency for you. Converting the kernel's data into a 
string is simple as code snippet above shows.

-Steve