auditd and redhat cluster

[Steve Grubb]

On Tuesday, March 01, 2016 02:57:45 PM Maupertuis Philippe wrote:
> The kernel is  : 2.6.32-573.12.1.el6.x86_64
> And the whole audit.rules file is  :

<snip>

> During the hour preceding the fence we got  these events from the passive
> node Key Summary Report
> ===========================
> total  key
> ===========================
> 891  system_commands (ping)
> 
> And on the active node  :
> 
> Key Summary Report
> ===========================
> total  key
> ===========================
> 1330  system_commands
> 286  deletion
> 
> I am going to follow your advice and to open a call with redhat.
> Anyway, I am interested in knowing if auditd has been reported to cause
> trouble without generating many events.

Those numbers work out to 27 events per minute. That's not really a lot of 
events. To see if its the rules or auditd causing the iowait, you might set 
the logging format to NOLOG. This will discard events rather than log them. If 
you still have iowait, its something to do with the rules. If that cleared it 
up, then auditd might be the source. Either way, put the format back to raw. 

I did some benchmarking of auditd over the holidays and posted some results 
here:

https://www.redhat.com/archives/linux-audit/2015-December/msg00061.html

I'd recommend:

flush = incremental
freq = 100

for a modest performance improvement.

-Steve


> -----Message d'origine-----
> De : Paul Moore [mailto:paul at paul-moore.com]
> Envoyé : mardi 1 mars 2016 14:25
> À : Maupertuis Philippe
> Cc : linux-audit at redhat.com
> Objet : Re: auditd and redhat cluster
> 
> On Mon, Feb 29, 2016 at 7:45 AM, Maupertuis Philippe 
<philippe.maupertuis at worldline.com> wrote:
> > Hi list,
> > 
> > One clusters fenced the passive node around two hours  after auditd
> > was started.
> > 
> > We have found that iowait has increased since auditd was started and
> > was unusually high.
> > 
> > Auditd wasn’t generating many messages and there were no noticeable
> > added activity on the disk were the audit and syslog files were written.
> > 
> > Besides watches, the only general rules were :
> > 
> > # creation
> > -a exit,always -F arch=b32 -S creat -S mkdir -S mknod -S link -S
> > symlink -S mkdirat -S mknodat -S linkat -S symlinkat -F uid=root -F
> > success=1 -k creation -a exit,always -F arch=b64 -S creat -S mkdir -S
> > mknod -S link -S symlink -S mkdirat -S mknodat -S linkat -S symlinkat
> > -F uid=root -F success=1 -k creation
> > 
> > # deletion
> > -a exit,always -F arch=b32 -S rmdir -S unlink -S unlinkat -F uid=root
> > -F
> > success=1 -k deletion
> > -a exit,always -F arch=b64 -S rmdir -S unlink -S unlinkat -F uid=root
> > -F
> > success=1 -k deletion
> > 
> > After the rebot we deleted all rules and didn’t notice extra iowait
> > anymore.
> > 
> > Could these rules be the cause of additional iowait even if not
> > generating many events (around 20 in two hours) ?
> > 
> > Is there any other auditd mechanism  that could explain this phenomenon ?
> > 
> > I would appreciate any hints.
> 
> Hi Philippe,
> 
> First, as this is a RH cluster product, I would suggest contacting RH
> support with your question if you haven't already; this list is primarily
> for upstream development and support.
> 
> If you are able to experiment with the system, or have a test environment, I
> would suggest trying to narrow down the list of audit rules/watches to see
> which rules/watches have the most affect on the iowait times.  You've
> listed four rules, but you didn't list the watches you have configured. 
> Also, what kernel version are you using?
> 
> --
> paul moore
> www.paul-moore.com
> 
> !!!*************************************************************************
> ************ "Ce message et les pièces jointes sont confidentiels et
> réservés à l'usage exclusif de ses destinataires. Il peut également être
> protégé par le secret professionnel. Si vous recevez ce message par erreur,
> merci d'en avertir immédiatement l'expéditeur et de le détruire.
> L'intégrité du message ne pouvant être assurée sur Internet, la
> responsabilité de Worldline ne pourra être recherchée quant au contenu de
> ce message. Bien que les meilleurs efforts soient faits pour maintenir
> cette transmission exempte de tout virus, l'expéditeur ne donne aucune
> garantie à cet égard et sa responsabilité ne saurait être recherchée pour
> tout dommage résultant d'un virus transmis.
> 
> This e-mail and the documents attached are confidential and intended solely
> for the addressee; it may also be privileged. If you receive this e-mail in
> error, please notify the sender immediately and destroy it. As its
> integrity cannot be secured on the Internet, the Worldline liability cannot
> be triggered for the message content. Although the sender endeavours to
> maintain a computer virus-free network, the sender does not warrant that
> this transmission is virus-free and will not be liable for any damages
> resulting from any virus transmitted.!!!"
> 
> --
> Linux-audit mailing list
> Linux-audit at redhat.com
> https://www.redhat.com/mailman/listinfo/linux-audit