Linux audit performance impact

Tue Feb 3 16:54:33 UTC 2015

Thanks for The info. But my question was rhetorical... I meant to say that
it would not be much... She is trying to bombard the system with open calls
... So lots and lots of events will be generated and kernel has to write
down the events some where or discard them...

On Tuesday, February 3, 2015, Richard Guy Briggs <rgb at redhat.com> wrote:

> On 15/02/03, Satish Chandra Kilaru wrote:
> > How many events can kernel accumulate without I/o ?
>
> The kernel default is 64 *buffers*, but I think Fedora and RHEL set it
> to 320.  It is now possible to set it to "0" which means limited only by
> system resources.  See "man auditctl", "-b" option.  An event can be
> made up of several buffers.
>
> Of course, how long a system lasts before the queue blows up depends on
> your rule set...
>
> However, at the moment, it will still write out to klog if auditd isn't
> running.
>
> > On Tuesday, February 3, 2015, Viswanath, Logeswari P (MCOU OSTL) <
> > logeswari.pv at hp.com <javascript:;>> wrote:
> >
> > > I don't want to disable auditing (i.e. disable audit record
> collection),
> > > but just do not want the records to delivered to user space since I
> want to
> > > remove the I/O overhead while running the performance test.
> > > Is there any option for this?
> > >
> > > -----Original Message-----
> > > From: Richard Guy Briggs [mailto:rgb at redhat.com <javascript:;>
> <javascript:;>]
> > > Sent: Thursday, January 29, 2015 10:23 PM
> > > To: Viswanath, Logeswari P (MCOU OSTL)
> > > Cc: Satish Chandra Kilaru; Steve Grubb; linux-audit at redhat.com
> <javascript:;>
> > > <javascript:;>
> > > Subject: Re: Linux audit performance impact
> > >
> > > On 15/01/29, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > > > Please read my question as “Is there any option to configure kaudit
> > > > not to log audit records to syslog? when auditd not running.”
> > >
> > > Yeah, remove audit=1 from the kernel command line, or set audit=0 in
> its
> > > place.  This will stop all but AVCs and if auditd has ever run since
> boot.
> > > If audit=0 is on the kernel boot line, it will be impossible to run
> auditd.
> > >
> > > There is a feature request that is likely coming soon that could be
> > > useful:
> > >
> > > https://bugzilla.redhat.com/show_bug.cgi?id=1160046
> > > "If no audit daemon is running, but an audit multicast subscriber is
> > > around, then the kernel shouldn't forward audit data to kmsg"
> > >
> > > > From: Viswanath, Logeswari P (MCOU OSTL)
> > > > Sent: Thursday, January 29, 2015 11:49 AM
> > > > To: 'Satish Chandra Kilaru'; Steve Grubb
> > > > Cc: linux-audit at redhat.com <javascript:;> <javascript:;>
> > > > Subject: RE: Linux audit performance impact
> > > >
> > > > Is there any option to configure kaudit not to log audit records to
> > > syslog when auditd is running?
> > > > This way we can assess the impact of enabling audit without involving
> > > disk I/o overhead.
> > > >
> > > > From: Satish Chandra Kilaru [mailto:iam.kilaru at gmail.com
> <javascript:;> <javascript:;>]
> > > > Sent: Thursday, January 29, 2015 9:12 AM
> > > > To: Steve Grubb
> > > > Cc: linux-audit at redhat.com <javascript:;> <javascript:;><mailto:
> linux-audit at redhat.com <javascript:;>
> > > <javascript:;>>; Viswanath,
> > > > Logeswari P (MCOU OSTL)
> > > > Subject: Re: Linux audit performance impact
> > > >
> > > > I agree with you... but writing to disk can trigger further events
> > > leading spiralling of events...
> > > > I brought down my server few times with stupid rules...
> > > >
> > > > On Wed, Jan 28, 2015 at 10:39 PM, Steve Grubb <sgrubb at redhat.com
> <javascript:;>
> > > <javascript:;><mailto:sgrubb at redhat.com <javascript:;>
> <javascript:;>>> wrote:
> > > > On Wednesday, January 28, 2015 10:18:47 AM Satish Chandra Kilaru
> wrote:
> > > > > Write your own program to receive audit events directly without
> > > > > using auditd...
> > > > > That should be faster ....
> > > > > Auditd will log the events to disk causing more I/o than u need...
> > > >
> > > > But even that is configurable in many ways. You can decide if you
> want
> > > > logging to disk or not and what kind of assurance that it made it to
> > > > disk and the priority of that audit daemon. Then you also have all
> the
> > > > normal tuning knobs for disk throughput that you would use for any
> > > > disk performance critical system.
> > > >
> > > > -Steve
> > > >
> > > > > On Wednesday, January 28, 2015, Viswanath, Logeswari P (MCOU OSTL)
> <
> > > > >
> > > > > logeswari.pv at hp.com <javascript:;> <javascript:;><mailto:
> logeswari.pv at hp.com <javascript:;>
> > > <javascript:;>>> wrote:
> > > > > >  Hi Steve,
> > > > > >
> > > > > > I am Logeswari working for HP.
> > > > > >
> > > > > >
> > > > > >
> > > > > > We want to know audit performance impact on RHEL and Suse linux
> to
> > > > > > help us evaluate linux audit as data source for our host based
> IDS.
> > > > > >
> > > > > > When we ran our own performance test with a test audispd plugin,
> > > > > > we found if a system can perform 200000 open/close system calls
> > > > > > per second without auditing, system can perform only 3000
> > > > > > open/close system calls auditing is enabled for open/close system
> > > > > > call which is a HUGE impact on the system performance. It would
> be
> > > > > > great if anyone can help us answering the following questions.
> > > > > >
> > > > > >
> > > > > >
> > > > > > 1)      Is this performance impact expected? If yes, what is the
> > > reason
> > > > > > behind it and can we fix it?
> > > > > >
> > > > > > 2)      Have anyone done any benchmarking for performance
> impact? If
> > > yes,
> > > > > > can you please share the numbers and also the steps/programs used
> > > > > > the run the same.
> > > > > >
> > > > > > 3)      Help us validating the performance test we have done in
> our
> > > test
> > > > > > setup using the steps mentioned along with the results attached.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Attached test program (loader.c) to invoke open and close system
> > > calls.
> > > > > >
> > > > > > Attached idskerndsp is the audispd plugin program.
> > > > > >
> > > > > > We used time command to determine how much time the system took
> to
> > > > > > complete 50000 open/close system calls without (results attached
> > > > > > Without-auditing) and with auditing enabled on the system
> > > > > > (With-auditing-NOLOG-audispd-plugin and With-auditing-RAW)
> > > > > >
> > > > > >
> > > > > >
> > > > > > System details:
> > > > > >
> > > > > >
> > > > > >
> > > > > > 1 CPU machine
> > > > > >
> > > > > >
> > > > > >
> > > > > > *OS Version*
> > > > > >
> > > > > > RHEL 6.5
> > > > > >
> > > > > >
> > > > > >
> > > > > > *Kernel Version*
> > > > > >
> > > > > > uname –r
> > > > > >
> > > > > > 2.6.32-431.el6.x86_64
> > > > > >
> > > > > >
> > > > > >
> > > > > > Note: auditd was occupying 35% of CPU and was sleeping for most
> of
> > > > > > the time whereas kauditd was occupying 20% of the CPU.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Thanks & Regards,
> > > > > >
> > > > > > Logeswari.
> > > >
> > > >
> > > >
> > > > --
> > > > Please Donate to www.wikipedia.org<http://www.wikipedia.org>
> > >
> > > > --
> > > > Linux-audit mailing list
> > > > Linux-audit at redhat.com <javascript:;> <javascript:;>
> > > > https://www.redhat.com/mailman/listinfo/linux-audit
> > >
> > >
> > > - RGB
> > >
> > > --
> > > Richard Guy Briggs <rbriggs at redhat.com <javascript:;> <javascript:;>>
> > > Senior Software Engineer, Kernel Security, AMER ENG Base Operating
> > > Systems, Red Hat Remote, Ottawa, Canada
> > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
> > >
> >
> >
> > --
> > Please Donate to www.wikipedia.org
>
> - RGB
>
> --
> Richard Guy Briggs <rbriggs at redhat.com <javascript:;>>
> Senior Software Engineer, Kernel Security, AMER ENG Base Operating
> Systems, Red Hat
> Remote, Ottawa, Canada
> Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
>

-- 
Please Donate to www.wikipedia.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/linux-audit/attachments/20150203/9e3661ef/attachment.htm>