Has anybody got any advice for the following problem? As I'm seeing some very odd behaviour with the auditd daemon in RHEL5.2 where under heavy system load the auditd process doesn't free any resources until all memory is consumed and the kernel kills the process with an Out Of Memory error.
The system I have is a heavily customised RHEL5.2 with some fairly stringent auditing rules specified, the config is attached. In addition to these rules there will be various SELinux AVCs being raised as well as events from my own software so the auditing system is kept quite busy, see the attached report.txt for the aureport summary .
I can reproduce this behaviour consistently by generating a heavy system load (CPU 100% usage) while also generating a significant number of audit events. After about 20 minutes the auditd process will have grown from 8Mb of memory to around 1Gb;
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3037 root 16 -3 2763m 921m 16 S 3.7 91.2 0:26.49 auditd
If the system is kept busy eventually auditd will consume all the memory available on the system and the process be killed by the kernel with an Out Of Memory error.
On the face of it this would seem to be a memory leak within auditd however once auditd has started to consume resources if the system load is reduced and therefore the number and frequency of audit events is reduced then auditd will free this memory and return back to the 8Mb memory usage. So it would seem that while auditd is kept busy it keeps hold of any resources it is allocating. Unfortunately I have systems that run under the level of load required to cause this issue fairly frequently.
The system is RHEL 5.2 and audit is version audit-1.7.17-3.
N E X O R
connect transform protect
Tel: +44 (0) 115 952 0500
Fax: +44 (0) 115 952 0519
Nexor is recognised as an Investor in People and is accredited to ISO9001/TickIT and ISO/IEC27001:2005. Further details of Nexor's accreditations can be found on our website.
DISCLAIMER: Privileged or confidential information may be contained in this message or within any files transmitted with it. If you are not the intended recipient, kindly destroy the message and notify the sender by reply email. Opinions, conclusions and other information in this message that do not relate to the official business of Nexor are neither given nor endorsed by it.
Nexor Limited, Bell House, Nottingham Science and Technology Park, University Boulevard, Nottingham, NG7 2RL
A company registered in England, No: 05152465
Number of changes in configuration: 301 Number of changes to accounts, groups, or roles: 0 Number of logins: 13 Number of failed logins: 14 Number of authentications: 24 Number of failed authentications: 2 Number of users: 2 Number of terminals: 15 Number of host names: 5 Number of executables: 120 Number of files: 348742 Number of AVC's: 23361 Number of MAC events: 1 Number of failed syscalls: 1101653 Number of anomaly events: 0 Number of responses to anomaly events: 0 Number of crypto events: 0 Number of keys: 0 Number of process IDs: 22880 Number of events: 1521282