Linux audit performance impact
Viswanath, Logeswari P (MCOU OSTL)
logeswari.pv at hp.com
Wed Jan 28 15:52:03 UTC 2015
Thanks for the quick reply.
Please look in-line for my replies.
From: Steve Grubb [mailto:sgrubb at redhat.com]
Sent: Wednesday, January 28, 2015 8:46 PM
To: linux-audit at redhat.com
Cc: Viswanath, Logeswari P (MCOU OSTL)
Subject: Re: Linux audit performance impact
On Wednesday, January 28, 2015 02:57:58 PM Viswanath, Logeswari P wrote:
> We want to know audit performance impact on RHEL and Suse linux to
> help us evaluate linux audit as data source for our host based IDS.
> When we ran our own performance test with a test audispd plugin, we
> found if a system can perform 200000 open/close system calls per
> second without auditing, system can perform only 3000 open/close
> system calls auditing is enabled for open/close system call which is a
> HUGE impact on the system performance. It would be great if anyone can help us answering the following questions.
> 1) Is this performance impact expected? If yes, what is the reason
> behind it and can we fix it?
I'll leave this for the kernel guys to answer. That said, I think more detailed information might be helpful.
If auditd is not started and events go to syslog, does the performance change?
To do this audit=1 on boot line and auditctl -R /etc/rules.d/your.rules
Logeswari=>System can perform 15000 open/close system calls per second which is better than earlier results.
what rules do you have loaded?
Logeswari=> # auditctl -l
LIST_RULES: exit,always syscall=open,close
What do you get when audit is enabled and no rules loaded?
Logeswari=> Impact is there but not major.
If you have other syscall rules loaded that are not open and openat or close, does the performance change? I suspect that if you trigger a rule, you are thrown onto the slow path. Open is perhaps the most lengthy because of multiple auxiliary records and path resolution. But we need data to tell.
Logeswari=> Yes, there is an major impact. I enabled write system call and this rule is first in the set of rules along with open/close.
That said, I know that the kernel audit path changed a couple years ago so it might be worthwhile to test against an old kernel to see if the change has affected performance.
Logeswari=> We tested with kernel 2.6.32. Should we test with old/new kernel?
> 2) Have anyone done any benchmarking for performance impact? If yes,
> can you please share the numbers and also the steps/programs used the
> run the same.
> 3) Help us validating the performance test we have done in our test
> setup using the steps mentioned along with the results attached.
> Attached test program (loader.c) to invoke open and close system calls.
> Attached idskerndsp is the audispd plugin program.
> We used time command to determine how much time the system took to
> 50000 open/close system calls without (results attached
> Without-auditing) and with auditing enabled on the system
> and With-auditing-RAW)
> System details:
> 1 CPU machine
> OS Version
> RHEL 6.5
> Kernel Version
> uname -r
> Note: auditd was occupying 35% of CPU and was sleeping for most of the
> time whereas kauditd was occupying 20% of the CPU.
> Thanks & Regards,
More information about the Linux-audit