content and format?

[John Calcote]

Steve,

On closer re-reading of this response, I find I have a few more questions:

>>> Steve Grubb <sgrubb at redhat.com> 12/14/06 11:25 AM >>>
On Thursday 14 December 2006 12:24, John Calcote wrote:
>> So what's in the future for linux audit regarding content and format?

> I think we should be in position to allow reformatting of audit information on 
> the fly early next year. I think the key to doing this as well as creating 
> many new tools will hinge on the audit parsing library.

> This library has been spec'ed out and designed with higher level languages in 
> mind. http://people.redhat.com/sgrubb/audit/audit-parse.txt The first problem 
> that anyone runs into if they want to make tools is how to parse the events. 
> This library will let you get past having to study all the messages to create 
> parsing rules.

I guess what I'm wondering is this: Why is a parser library even necessary? I'm not questioning your intent - I'm wondering what the rationale is behind such a library. Is it that you're trying to find a way of not defining ANY formatting rules or event taxonomy (standardized event type hierarchy) so the system will be as useful as possible to as many domains as possible?

If so, great! This is a wonderful ideal, but is it really necessary? I mean, this is a fairly new system, and everyone understands that it's being defined (by this community) right now. Isn't this a great opportunity (really the only opportunity) to impose some reasonable amount of message and event type structure so that everyone get's the incredible advantages of standardized structured audit data? A reasonable amount of structure and taxonomy information also alleviates about 80 percent of the need for a parsing library, as such a library would primarily be focused on returning standard field values - it becomes almost trivial. But more importantly - it becomes very accurate - which (IMHO) is paramount in an auditing system.

> The audit daemon has been created with a realtime interface so that other 
> analytical programs can get their hands on the data in near realtime. This 
> offers a lot of advantages over cron based techniques that read from a file. 
> The realtime interface lets the daemon itself be simple so that it can pass a 
> CAPP/LSPP eval and yet offer expansion capabilities.
> 
> The plan to allow other formats, reactive programs, or centralized logging is 
> to create a dispatcher that reads the output of the daemon and hands the data 
> to programs that have subscribed to it. Right now, we have a primitive 
> dispatcher to test the concept out with SE Linux where a program analyzes 
> events and offers help to users if they see a pattern that would suggest a 
> boolean needs to be changed.

> There is another dispatcher that is close to what I am thinking of:
> http://www.linuon.com/dowloads/led/ 

> Anyways, what we can do is have a plugin that takes audit events and uses the 
> parser library to extract the fields its needs for a message and then write 
> it to disk or send it across the network.

> John, would this scheme work for you?

This sounds great! Please don't take my comments as derisive - I mean only to provide constructive input. I totally recognize you guys are the experts in this area, and I certainly don't claim to understand half of what you're talking about on this list with respect to transport, performance, bandwidth and efficiency.

I've spent a lot of my time at a much higher level - CIO level in corporations clambering for better auditing tools. Maybe we're trying to solve two different problems - I don't know - but one thing I'm very aware of is that a system that allows AUTOMATIC analysis of audit logs with a high degree of accuracy is one that will make a LOT of IT departments happy.

I'm also open to the (strong) possibility that I'm missing the bigger picture. My take on quality automatic analysis revolves primarily around a standard imposed event format and taxonomy - perhaps one enforced by the logging UI, perhaps not. In either case, however, a small amount of standardized structure and event taxonomy has a great many benefits in the world of audit log auto-analysis tools.

What other options might I be missing? Any responses would be appreciated greatly.

Thanks!
--John