Audit / Netlink slowness

[Bernardo Innocenti]

Steve G wrote:


>>Running "strace -r 2>strace.out su", I discovered that
>>netlink communication is the major cause of slowdown.
> 
> netlink in theory should be fast. No routing or collisions.

I suppose so, but I don't know any other netlink user
I could compare with to make sure it's really a problem
specific to audit.


>>"su" connects to a NETLINK_AUDIT socket 3 or 4 times.
>>Each time it does 2 sendto() + recvfrom() operations,
> 
> It does an audit subsystem status to see if its enabled and if so a send of
> auditable information. What version of pam, su, audit-libs, kernel are you using?

 pam-0.79-10
 audit-libs-0.9.3-1
 coreutils-5.2.1-31
 kernel-2.6.11-1.1240_FC4

SELinux is disabled.

The kernel is a custom build with several drivers removed and
some debugging options turned off.  Should perform better than
stock kernels.


>>with a latency of ~200ms.  This adds up to 800ms wasted
>>time.
> 
> Just out of curiosity, what cpu & clock speed do you have?

It's an Athlon 2500 (3547.13 bogomips)


> Are you running UP or SMP kernel?

UP, with preemption disabled.


> This code path should be entirely cpu bound as no io devices are
> involved.

Does the audit socket communication talk synchronously with
auditd?  If so, the problem could be there too.


>>Disabling CONFIG_AUDIT in the kernel makes su and ssh
>>very fast again.
> 
> 
> You also lose part of your SE Linux avc messages. There was a deadlock condition
> discovered and reported on the NSA SE Linux mail list. The solution was to move
> part of the processing to syscall exit audit processing. With audit not compiled
> in or enabled, you get an abreviated avc message under some conditions.

I also disabled SELinux, mainly because I wasn't willing to
fix all my services to run properly with the strict policy that
was initially shipped with FC2.  Then I just didn't find the
time/motivation to turn it on again.  Yes, lame me :-)


>>Is this behavior to be expected?
> 
> Not exactly. There will be a *some* delay as we've added a lot of new
> functionality, but 800 ms total delay is excessive. I'll see if we can find the
> culprit.

Could you please also try an "strace -r" to make sure I'm
not the only one seeing this?

-- 
  // Bernardo Innocenti - Develer S.r.l., R&D dept.
\X/  http://www.develer.com/