content and format?

[Steve Grubb]

On Tuesday 02 January 2007 17:12, John Calcote wrote:
> > This library will let you get past having to study all the messages to
> > create parsing rules. 
>
> I guess what I'm wondering is this: Why is a parser library even necessary?

So that you can disassemble the incoming event and rewrite it the way you 
want.

> I'm not questioning your intent - I'm wondering what the rationale is
> behind such a library. Is it that you're trying to find a way of not
> defining ANY formatting rules or event taxonomy (standardized event type
> hierarchy) so the system will be as useful as possible to as many domains
> as possible?

Sort of. The current audit system provides more information than many people 
would like. (Do you really need fsgid? probably not). Its not necessary to 
keep all this information if you don't need it. Or if you'd rather have it 
written in an order that's more intuitive to you, or compatible with a 
certain protocol. The first step is to decompose the native event format and 
then you are free to restructure it to your liking.

> If so, great! This is a wonderful ideal, but is it really necessary?

I think so. You might not want subj/obj labeling but someone else might. Same 
with any other attribute.

> I mean, this is a fairly new system, and everyone understands that it's
> being defined (by this community) right now. Isn't this a great opportunity
> (really the only opportunity) to impose some reasonable amount of message
> and event type structure so that everyone get's the incredible advantages
> of standardized structured audit data?

The current audit events are structured. At some point in the future, I'd like 
to allow for binary event data. The parsing library is is a step in that 
direction by providing a layer of abstraction from the low level audit record 
formatting. Analytical apps should not concern themselves with the details of 
the format on disk or buffer.

> A reasonable amount of structure and taxonomy information also alleviates
> about 80 percent of the need for a parsing library, as such a library would
> primarily be focused on returning standard field values - it becomes almost
> trivial.

That's what I'm building and yes it is trivial.

> But more importantly -  it becomes very accurate - which (IMHO) is paramount
> in an auditing system.

The current audit system should be very accurate. It provides more information 
than you probably need.

> > John, would this scheme work for you?
>
> This sounds great! Please don't take my comments as derisive - I mean only
> to provide constructive input.

Sure.

> I've spent a lot of my time at a much higher level - CIO level in
> corporations clambering for better auditing tools.

We'll get there. Its just taking a long time since I have lots of other things 
to do besides this.

> Maybe we're trying to solve two different problems - I don't know - but one
> thing I'm very aware of is that a system that allows AUTOMATIC analysis of
> audit logs with a high degree of accuracy is one that will make a LOT of IT
> departments happy.

No, we're not solving different problems. The intent of this design is to 
allow it to meet audit level 9 in dcid 6/3. It will eventually do real time 
response to threats. I also look at SOX and FIPS-200 requirements for future 
direction.

> I'm also open to the (strong) possibility that I'm missing the bigger
> picture.

Could be. I don't have much time to write papers about what this thing will do 
or what its current capabilities are. But progress is steady towards better 
event information, flexibility, and analysis. This mail list is probably the 
best way to find out the future plans and how it works until I (or others) 
have a chance to document it.

> My take on quality automatic analysis revolves primarily around a 
> standard imposed event format and taxonomy 

This should be trivial. It should be mostly a reorg of data in events. There 
may be some things not required by CAPP/LSPP that people would like the audit 
system to collect. We can add those when they are identified. Can you give me 
an example of a standard event? We can map it onto what the current audit 
system is providing to see if anything is missing.

-Steve