Offline audit trail analysis
Wieprecht, Karen M.
Karen.Wieprecht at jhuapl.edu
Tue Sep 11 20:59:07 UTC 2007
This isn't probably as savvy as some of you would like, but I started
writing it before there were any auparse libraries to help with this
kind of stuff. I've done similar stuff on Solaris, Mac OSX, irix,
etc. My linux version scripting isn't finished (not enough hours in
the day), but it's still useful for my log reviews. At some point I
hope to get out internal people to bless it enough to share with whoever
might be interested in the open source world, but I'm not there yet.
here's the general thing I've setting up:
I use scripting to do the following on each machine :
- rotate the log
- (I want to try a script segment to make sure all related audit event
records are not interleaved)
- run an ausearch on it with some flags (interpret, all messages) and
some shell commands to get all related records for a particular event
onto a single line
- save this processed file (with user IDs, etc. particular to that
system) as something like audit.log.hostname.2006-09-10.ausearched
- Store the original and "ausearched" files in a central place so I have
them all gathered for my weekly log review
Then weekly, I use a "reviewlogs" script to post process the
"ausearched" "single line" data into something more readable. It also
filters some noise I don't care about. The Chronological results look
like this from various test sessions:
karen FAILED TO mv /etc/nsswitch.conf /etc/nsswitch.conf.mine
on patton, Permission denied on 01/15/2007 14:05:16.582
dan FAILED TO console login into patton on 01/17/2007
13:21:45.133
dan successful console login into patton on 01/17/2007
13:21:59.152
dan successful console logout from patton on
01/17/2007 13:22:26.059
karen FAILED TO ssh login into patton from ? on 01/17/2007
13:34:26.517
karen successful ssh login into patton from
patton.jhuapl.edu on 01/17/2007 13:34:31.387
karen FAILED TO touch /etc/nsswitch.conf on patton,
Permission denied on 01/17/2007 13:34:57.759
karen FAILED TO touch /etc/nsswitch.conf on patton,
Permission denied on 01/17/2007 13:34:57.759
karen successful SU LOGIN (PAM authentication) as root
on patton on 01/17/2007 13:35:15.140
karen successful SU LOGIN (PAM accounting) as root
on patton on 01/17/2007 13:35:15.141
karen successful SU LOGIN (PAM session open) as root
on patton on 01/17/2007 13:35:15.141
-- karen as root -- successful SU LOGIN (PAM authentication)
as dan on patton on 01/17/2007 13:35:31.364
-- karen as root -- successful SU LOGIN (PAM accounting)
as dan on patton on 01/17/2007 13:35:31.366
-- karen as root -- successful SU LOGIN (PAM session open)
as dan on patton on 01/17/2007 13:35:31.367
-- karen as dan -- FAILED TO rm /etc/hosts on
patton, Permission denied on 01/17/2007 13:35:39.129
-- karen as dan -- FAILED TO rm /etc/hosts on
patton, Permission denied on 01/17/2007 13:35:40.680
-- karen as root -- successful SU LOGIN (PAM setcred)
as dan on patton on 01/17/2007 13:35:44.177
-- karen as root -- successful CLOSED SU as dan on
patton on 01/17/2007 13:35:44.178
karen successful SU LOGIN (PAM setcred) as root
on patton on 01/17/2007 13:35:46.089
karen successful CLOSED SU as root on patton on 01/17/2007
13:35:46.089
Note that it's easy to spot users trying to make it look like other
users tried to do stuff. When I review the logs, I usually sort the
reformatted lines so that similar entries are all grouped (which makes
it really easy to let chunks of stuff I don't care about scroll by
quickly). If I see something suspicious, I'll go back and look at the
stuff in chronological order to see what was going on. Here's what the
above sample looks like sorted (and if I had a lot of data, the chunks
of similar stuff would be more apparent):
-- karen as dan -- FAILED TO rm /etc/hosts on
patton, Permission denied on 01/17/2007 13:35:39.129
-- karen as dan -- FAILED TO rm /etc/hosts on
patton, Permission denied on 01/17/2007 13:35:40.680
-- karen as root -- successful CLOSED SU as dan on
patton on 01/17/2007 13:35:44.178
-- karen as root -- successful SU LOGIN (PAM accounting)
as dan on patton on 01/17/2007 13:35:31.366
-- karen as root -- successful SU LOGIN (PAM authentication)
as dan on patton on 01/17/2007 13:35:31.364
-- karen as root -- successful SU LOGIN (PAM session open)
as dan on patton on 01/17/2007 13:35:31.367
-- karen as root -- successful SU LOGIN (PAM setcred)
as dan on patton on 01/17/2007 13:35:44.177
dan FAILED TO console login into patton on 01/17/2007
13:21:45.133
dan successful console login into patton on 01/17/2007
13:21:59.152
dan successful console logout from patton on
01/17/2007 13:22:26.059
karen FAILED TO mv /etc/nsswitch.conf /etc/nsswitch.conf.mine
on patton, Permission denied on 01/15/2007 14:05:16.582
karen FAILED TO ssh login into patton from ? on 01/17/2007
13:34:26.517
karen FAILED TO touch /etc/nsswitch.conf on patton,
Permission denied on 01/17/2007 13:34:57.759
karen FAILED TO touch /etc/nsswitch.conf on patton,
Permission denied on 01/17/2007 13:34:57.759
karen successful CLOSED SU as root on patton on 01/17/2007
13:35:46.089
karen successful SU LOGIN (PAM accounting) as root
on patton on 01/17/2007 13:35:15.141
karen successful SU LOGIN (PAM authentication) as root
on patton on 01/17/2007 13:35:15.140
karen successful SU LOGIN (PAM session open) as root
on patton on 01/17/2007 13:35:15.141
karen successful SU LOGIN (PAM setcred) as root
on patton on 01/17/2007 13:35:46.089
karen successful ssh login into patton from
patton.jhuapl.edu on 01/17/2007 13:34:31.387
We don't have a ton of Linux systems (yet), so this strategy works well
for reviewing a handful of systems from a central location. The
scripts save me a lot of time interpretting any one given log entry as
well as keeping me from having to log into N systems one at a time and
running M commands on each one each to decide if someone was messing
around on my machines or not.
Karen Wieprecht
________________________________
From: linux-audit-bounces at redhat.com
[mailto:linux-audit-bounces at redhat.com] On Behalf Of Todd, Charles
Sent: Tuesday, September 11, 2007 3:32 PM
To: linux-audit at redhat.com
Subject: Offline audit trail analysis
Thanks to Steve for being our biggest target for questions on this list!
Has anyone talked about sane ways to do offline analysis of Linux audit
logs? Presumably, this would be on another Linux system, but maybe not
the same host, and probably not on the same release or with the same
username/IP address access. Conceptually, ausearch would save and
optionally read a system's "configuration" to be saved for
interpretation later.
My goal is central logging, but doing the reporting/analysis on the
central host. That way, I can see a user across the Enterprise (or at
least in the Linux hosts), but with all the power of ausearch for
refining the report. Ideally, I would do an ausearch -ts <date> -te
<date> --raw --config-to=<hostname.ausearch.config> and it would do
things like saving the syscall lookup table, lookup users referenced in
the reported audit trail, and resolve IP addresses references in the
reported audit trail. Maybe one config file could be written for each
data type in an existing format (e.g. users in /etc/passwd format, hosts
in /etc/hosts format, etc.). I'm mainly after whether or not anyone has
considered extending ausearch for this kind of processing?
This way, an archive of raw logs could be kept along with the exact
system configuration which allows offloading the audit trail analysis to
a trusted location, rather than risk side effects from a rootkit.
Charlie Todd
Ball Aerospace & Technologies Corp.
This message and any enclosures are intended only for the addressee.
Please
notify the sender by email if you are not the intended recipient. If
you are
not the intended recipient, you may not use, copy, disclose, or
distribute this
message or its contents or enclosures to any other person and any such
actions
may be unlawful. Ball reserves the right to monitor and review all
messages
and enclosures sent to or from this email address.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/linux-audit/attachments/20070911/e9d39c38/attachment.htm>
More information about the Linux-audit
mailing list