Odd memory usage in auditd

[Ross Kirk]

Hi,

 

Has anybody got any advice for the following problem? As I'm seeing some very odd behaviour with the auditd daemon in RHEL5.2 where under heavy system load the auditd process doesn't free any resources until all memory is consumed and the kernel kills the process with an Out Of Memory error.

 

The system I have is a heavily customised RHEL5.2 with some fairly stringent auditing rules specified, the config is attached. In addition to these rules there will be various SELinux AVCs being raised as well as events from my own software so the auditing system is kept quite busy, see the attached report.txt for the aureport summary .

 

I can reproduce this behaviour consistently by generating a heavy system load (CPU 100% usage) while also generating a significant number of audit events. After about 20 minutes the auditd process will have grown from 8Mb of memory to around 1Gb;

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND    

3037 root      16  -3 2763m 921m   16 S  3.7 91.2   0:26.49 auditd

If the system is kept busy eventually auditd will consume all the memory available on the system and the process be killed by the kernel with an Out Of Memory error.

 

On the face of it this would seem to be a memory leak within auditd however once auditd has started to consume resources if the system load is reduced and therefore the number and frequency of audit events is reduced then auditd will free this memory and return back to the 8Mb memory usage. So it would seem that while auditd is kept busy it keeps hold of any resources it is allocating. Unfortunately I have systems that run under the level of load required to cause this issue fairly frequently.

 

The system is RHEL 5.2 and audit is version audit-1.7.17-3.

 

Regards

Ross Kirk

Software Engineer

N E X O R

connect  transform  protect

Tel:  +44 (0) 115 952 0500

Fax: +44 (0) 115 952 0519

mailto:ross.kirk at nexor.com <mailto:ross.kirk at nexor.com> 

http://www.nexor.com <http://www.nexor.com> 

Nexor is recognised as an Investor in People and is accredited to ISO9001/TickIT and ISO/IEC27001:2005. Further details of Nexor's accreditations can be found on our website.

DISCLAIMER: Privileged or confidential information may be contained in this message or within any files transmitted with it. If you are not the intended recipient, kindly destroy the message and notify the sender by reply email. Opinions, conclusions and other information in this message that do not relate to the official business of Nexor are neither given nor endorsed by it.

Nexor Limited, Bell House, Nottingham Science and Technology Park, University Boulevard, Nottingham, NG7 2RL

A company registered in England, No: 05152465

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/linux-audit/attachments/20101007/3ced4377/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: audit.rules
Type: application/octet-stream
Size: 2165 bytes
Desc: audit.rules
URL: <http://listman.redhat.com/archives/linux-audit/attachments/20101007/3ced4377/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: auditd.conf
Type: application/octet-stream
Size: 424 bytes
Desc: auditd.conf
URL: <http://listman.redhat.com/archives/linux-audit/attachments/20101007/3ced4377/attachment-0001.obj>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: report.txt
URL: <http://listman.redhat.com/archives/linux-audit/attachments/20101007/3ced4377/attachment.txt>