Linux audit performance impact

[Casey Schaufler]

On 2/18/2015 1:49 PM, Paul Moore wrote:
> On Wed, Feb 18, 2015 at 4:13 PM, Richard Guy Briggs <rgb at redhat.com> wrote:
>> On 15/02/17, Viswanath, Logeswari P (MCOU OSTL) wrote:
>>> I agree that changing the formatting of the records could break the existing applications
>>> that consume them, and I didn't mean changing or eliminating of the formatting completely.
>>> We agree that formatting is required for logging the records(as buffers) into the log files.
>>> We are wondering if these records can be made available as RAW records so that the
>>> analytical programs which are capable of reading them for processing can perform better.
>> There are tools that completely ignore any of the audit userspace suite
>> including libaudit, so changing the formatting in the kernel and
>> deferring to userspace to later do that formatting is not currently an
>> option.
> It is if you take a versioned API approach where the kernel defaults
> to the current behavior and switches, per-socket/connection, at the
> request of userspace.  It's really the only way to have a graceful
> transition with audit.
>
>>> This option of RAW mode for the events can be an additional option
>>> where, kauditd delivers the audit buffer without formatting. Any
>>> comments on this?
>> For a transition period if we were to consider it, it would mean
>> rewriting *all* places in the kernel that generate audit messages and
>> provide two paths switched on this RAW mode for each one of them, then
>> copying all that duplication to userspace libaudit.
> Your comment is a little vague, so let me mention what I'm currently
> considering: we convert all of the in-kernel audit users away from
> generating strings in the context of the caller, instead having them
> record information in a native/struct/etc. format that would be later
> used by the kernel audit subsystem to generate the audit records (in
> whatever format(s) is(are) requested).  This actually has advantages
> beyond the record format work, it moves the issue of record formatting
> (always a problem) out of the caller and into audit itself which
> should hopefully prevent future audit abuses (a netlink attribute
> based record format would likely help further).

The existing audit system is pretty hard on the security modules, too.
An internal structure that captures the information and formats it later
makes a whole lot of sense provided the information required to do the
formatting is available at that later time. It also allows for flexibility
in adding new information to audit records. A new security module could
add information it considers "security relevant" that other modules don't
without mucking up the audit records from existing modules.

In Irix (The kids on the list can look that up elsewhere :) ) audit
data was gathered as a collection of audit tokens, each of which
contained a chuck of information such as the MLS label, or the DAC
attributes of a process. The tokens were combined to create a complete
record late in the processing. The scheme didn't require much in the
way of structure.

I've done several audit systems and would be happy to contribute
to a revision of the Linux implementation.

>
>> According to Linus' decree, it would need to remain that way until we
>> were certain that all tools including ones we don't know about had
>> switched over.
> I would imagine a scenario where we introduced the new format in stages:
>
> #1 - Move in-kernel audit record string generation completely into
> kernel/audit*.c.  Benefits everyone regardless of the audit format.
>
> #2 - Introduce a versioned audit API.  The most difficult step for
> obvious reasons.
>
> #3 - Deprecate the old/existing audit record format, make it a Kconfig
> option that defaults to off and emit a warning when the old formatting
> is used.  This will be a year, and most likely more, after step #2.
>
> #4 - Remove the old/existing audit record code.  Once again, this
> would happen a couple of years after step #3.
>
> However, nothing is really determined yet, this is just my current thinking.
>