Patch to auparse to handle out of order messages 3 of 3

[Paul Moore]

On Fri, Jan 8, 2016 at 2:27 AM, Burn Alting <burn at swtf.dyndns.org> wrote:
> On Thu, 2016-01-07 at 22:06 -0500, Paul Moore wrote:
>> On January 7, 2016 6:47:02 PM Steve Grubb <sgrubb at redhat.com> wrote:
>>
>> > On Friday, January 08, 2016 10:05:13 AM Burn Alting wrote:
>> >> Steve,
>> >>
>> >> Can I suggest you modify src/ausearch-lol.c:check_events() to add in the
>> >> AUDIT_PROCTITLE check (will reduce memory overhead as events will be
>> >> flushed faster).
>> >
>> > OK. Good suggestion. The SVN repo has been updated.
>> >
>> >
>> >> Also can we ask Richard put a comment into the appropriate location in
>> >> the kernel code to indicate the link between ausearch/aurport/auparse
>> >> depending on AUDIT_PROCTITLE being the last record of an event if
>> >> present.
>> >
>> > I'll let them answer.
>>
>> Good thing I happened to read this message, I had stopped reading this
>> thread...
>>
>> I really dislike comment only patches and I really, really dislike the
>> fixed format fields/records/etc. that permeates so much of audit these
>> days.  I'll reserve final judgement for if/when any patches are posted, but
>> just to be clear, I'm not very excited about stuff like this.
>
> This is just a request to the kernel audit team, to note that the user
> level audit capability is making use of the AUDIT_PROCTITLE record to be
> an end of event marker. If you believe this is an unacceptable risk for
> downstream processing, then we can take this out and hence withdraw the
> request. The alternative is to maintain status quo, and/or optionally
> emit the AUDIT_EOE record into the stored audit and be done with it, and
> accept the storage cost.

I've already ranted plenty about field ordering so I won't bore you
all again, but you can apply the same rants to record ordering for an
event.  I'm begrudgingly going to accept the field ordering in the
current fixed-string format, but that will go away in the future.  I'm
much, much less inclined to accept the record ordering, especially
given the idea that there is talk of optionally removing the
AUDIT_PROCTITLE record.

> I wholeheartedly agree about the challenges we have with respect to the
> current format of the audit events emitted by the kernel. I spend a lot
> of effort converting the unstructured, sometimes inconsistently
> displayed events into more structured data.
>
> I believe the way forward is to define a more correct, efficient AND
> extensible output form for the kernel. On the user space side, we assist
> the existing consumers of our audit data (our customers so to speak) by
> providing a legacy audispd plugin to take the 'refined' data and format
> it into the current 'mash-up'. To assist this interim measure/plugin,
> and state that is IS interim, when converting the kernel code to emit a
> record, ensure our new method can record the old/legacy format in some
> way. The legacy formatting should be able to be compiled out.

Due to other projects, it is taking me much longer than anticipated,
but I have no given up on my effort to arrive at a new
kernel/userspace format.  I started some work towards removing
audit_log_format() before the holidays, which is the important first
step.

-- 
paul moore
www.paul-moore.com