Convert audit log to JSON/XML with aushape

[Nikolai Kondrashov]

Hi everyone,

I would like to introduce a tool I have been working on for a while with
Steven Grubb's guidance. It's an audit log converter called "aushape".
It is based on auparse and can convert raw audit log to JSON and XML.

     https://github.com/Scribery/aushape

Aushape can be used standalone, or as an audispd plugin for on-the-fly
conversion.

The output schema is being designed to correspond closely to original log
structure, but to be event-, rather than record-oriented. I.e. the log
consists of a series of events, with each containing one or more records.

Ultimately, the schema and part of the conversion code will be generated from
the official record and field dictionaries, and the intent is to have aushape
a part of auditd distribution.

I would like to build something that will have more uses than just within Red
Hat projects I'm working on, so I ask you to please take a look at aushape,
its interface and output schemas, tell me what you think about it, if you can
use it, and what you would like changed. I will be glad to answer any
questions you might have.

One of the aims for me is to be able to store the converted audit log in
ElasticSearch and query it from there easily with the help of Kibana. That
puts some limits on the output structure. One of them is that an output event
can only contain unique record types, e.g. it can't contain repeated "PATH" or
"EXECVE" records. Instead, those repeated records are aggregated and stored as
an array under the correponding record type container. "EXECVE" records are
decoded into a simple argument list and "PATH" records are sorted by their
item ID, in particular. This is done to simplify matching on specific record
fields in ElasticSearch and to avoid using "nested" type, which Kibana has
difficulties with.

At the moment aushape output includes raw log messages it was generated from
to help with debugging, but eventually that will be optional. Features coming
soon are limiting event and record sizes (they can get very big
theoretically), and reporting any conversion errors in-band, as special kind
of events, preserving the original log lines, so they could be re-processed.

Nick