[redhat-lspp] Linux Audit Performance Analysis
George Wilson
gcwilson at us.ibm.com
Tue Apr 18 01:17:35 UTC 2006
Following is an analysis by Jose Santos. Thanks very much for the help,
Jose.
--
George
We were able to reproduce the on the machine that George set up for us and
we able to look into the problem in more detail. At 100 rules we spend
about 35% of our time in audit_filter_rules() and 16% in
audit_filter_syscall. While most of the time is spent in
audit_filter_rules(), the real problem is in a check in
audit_filter_syscall() that goes inside a list_for_each_entry_rcu() which
call audit_filter_rules()
list_for_each_entry_rcu(e, list, list) {
if ((e->rule.mask[word] & bit) == bit
---> && audit_filter_rules(tsk,
&e->rule, ctx, &state))
While I don't have a deep understanding of how the auditing code works, it
looks like this list has one element per audit rule and as we increase the
number of rules, the number of times we call audit_filter_rules()
increases. There is some optimization that could be done to optimize the
performance audit_filter_rules(), but the key is to avoid calling this
function within the loop on audit_filter_syscall(). My suggestion would
be to eliminate the list completely and replace it with something that
requires a lot less searching such as a hash table. Since it seems like
this list would only be re-hashed when "auditctl -R" is invoked (not very
often I would assume), a decent hashing algorithm that does a good job of
avoiding collisions can be design without adding to much overhead when
compared to the current implementation.
As far as optimizing audit_filter_rules(), the first thing I would do
would be to in-line that function. From the Oprofile data that we have,
it seems like we could gain between 4 to 7%. Most of the time spent in
this function is inside the switch case (about 12%). In order to help the
compiler we could we could split the audit types into categories to avoid
doing the checking for all those 27 case statements each time. This
switch is also with in a for loop and the break only counts for the switch
statement, not for the loop. This loop will continue even if "results"
has been assigned a value. Instead of break, wouldn't a "goto out;" work
better for this case. Again, I know nothing of the audit call.
Once the issues with audit_filter_syscall() are fixed, changes to
audit_filter_rules() should have less of an impact so my focus would be
here first.
*********************************************************************************
Jose R. Santos
LTC - Performance
Thanks,
George Wilson
IBM LTC Security Development
More information about the redhat-lspp
mailing list