New draft standards

[Steve Grubb]

On Thursday, December 10, 2015 12:40:55 PM F Rafi wrote:
> My comments are more from a log user (not developer) perspective. We are
> exporting close to 10GB/day of mostly auditd logs. This will potentially go
> upto 20GB/day next year.
> 
> I'd prefer the ability to translate all auditd logs before they are written
> to disk. I believe this is what you have proposed, just wanted to confirm.

That is not exactly what I proposed. What I was proposing was to record the 
translation of things that could change between systems and thus prevent 
correct interpretation later. Doing all translations is technically possible 
but would slow down auditd just a bit and increase the amount of data on disk. 
But doing this is not really necessary for the native audit tools.

But I guess this gives me an opportunity to ask the community what tools they 
are using for audit log collection and viewing? Its been a couple years since 
e had this discussion on the mail list and I think some things have changed.

Do people use ELK?
Apache Flume?
Something else?

It might be possible to write a plugin to translate the audit logs into the 
native format of these tools.


> This means that uid / gid / auid will resolve on the same machine that the
> logs were generated on.
> 
> You mentioned IP translation in the enrichment doc. I'm currently tackling
> this for automated analysis of these logs (syscall 42). Currently the IPs
> are written in hex. It would be nice to have this translated into the IPv4
> decimal octets before the logs are written (I haven't checked what is
> written for IPv6).
> 
> One thing I'm having a very hard time reconciling are the multiple lines
> per log event.

In a way, that can't be avoided. I am working on a new representation of the 
audit stream that would be just one line in the "cooked" mode.

-Steve