[redhat-lspp] Linux Audit Performance Analysis

George Wilson gcwilson at us.ibm.com
Tue Apr 18 01:17:35 UTC 2006


Following is an analysis by Jose Santos.  Thanks very much for the help, 
Jose.

--
George


We were able to reproduce the on the machine that George set up for us and 
we able to look into the problem in more detail.  At 100 rules we spend 
about 35% of our time in audit_filter_rules() and 16% in 
audit_filter_syscall.  While most of the time is spent in 
audit_filter_rules(), the real problem is in a check in 
audit_filter_syscall() that goes inside a list_for_each_entry_rcu() which 
call audit_filter_rules()

                list_for_each_entry_rcu(e, list, list) {
                        if ((e->rule.mask[word] & bit) == bit
--->                                     && audit_filter_rules(tsk, 
&e->rule, ctx, &state))

While I don't have a deep understanding of how the auditing code works, it 
looks like this list has one element per audit rule and as we increase the 
number of rules, the number of times we call audit_filter_rules() 
increases.  There is some optimization that could be done to optimize the 
performance audit_filter_rules(), but the key is to avoid calling this 
function within the loop on audit_filter_syscall().  My suggestion would 
be to eliminate the list completely and replace it with something that 
requires a lot less searching such as a hash table.  Since it seems like 
this list would only be re-hashed when "auditctl -R" is invoked (not very 
often I would assume), a decent hashing algorithm that does a good job of 
avoiding collisions can be design without adding to much overhead when 
compared to the current implementation.

As far as optimizing audit_filter_rules(), the first thing I would do 
would be to in-line that function.  From the Oprofile data that we have, 
it seems like we could gain between 4 to 7%.  Most of the time spent in 
this function is inside the switch case (about 12%).  In order to help the 
compiler we could we could split the audit types into categories to avoid 
doing the checking for all those 27 case statements each time.  This 
switch is also with in a for loop and the break only counts for the switch 
statement, not for the loop.  This loop will continue even if "results" 
has been assigned a value.  Instead of break, wouldn't a "goto out;" work 
better for this case.  Again, I know nothing of the audit call.

Once the issues with audit_filter_syscall() are fixed,  changes to 
audit_filter_rules() should have less of an impact so my focus would be 
here first.

*********************************************************************************
Jose R. Santos
LTC - Performance


Thanks,
George Wilson
IBM LTC Security Development




More information about the redhat-lspp mailing list