Maximum Value for q_depth

[Steve Grubb]

Hello,

On Wednesday, December 8, 2021 11:00:50 PM EST Amjad Gabbar wrote:
> Got it. Makes sense to me. Thanks for the explanation Steve.
> 
> One last thing though based on the discussion we had, if the kernel is able
> to offload events even during bursts, wouldn’t setting q_depth
> =backlog_limit be enough?

Depends on the client attached to the af_unix socket. The kernel could have a 
burst and then another. If the client isn't reading the af_unix socket fast 
enough, there could still be  overflows. That said, it is likely to be enough. 
But I'd look into the client app that reads the af_unix socket to see why 
it's taking so long.

> The reason being if there was an overflow on the kernel side, a different
> message would be printed in the logs but because it is all dispatch errors,
> I assume the kernel is able to handle the burst which is why the reasoning
> of increasing q_depth to backlog_limit.

The q_depth being 90 is your likely problem. Realistically, that only allows 
20 - 30 events to be enqueued.

-Steve

> On Wed, Dec 8, 2021 at 4:38 PM Steve Grubb <sgrubb at redhat.com> wrote:
> > On Wednesday, December 8, 2021 4:54:18 PM EST Amjad Gabbar wrote:
> > > 1. The version of auditd is 1:2.8.4-3 and the plugins are af_unix.conf
> > 
> > and
> > 
> > > syslog.conf for audisp. The q_depth is currently set to 80 and I think
> > > it
> > > calls for an increase but not sure if there is a way to figure out what
> > 
> > the
> > 
> > > proper number would be?
> > 
> > There is no good calculation that I can give you. It depends on the
> > average
> > rate of incoming events and the rate that they can be offloaded to the
> > plugins
> > + some margin in case there is a burst. Looking at the 2.8.5 code, the
> > default is 250.
> > 
> > https://github.com/linux-audit/audit-userspace/blob/2.8_maintenance/init.
> > d/ audispd.conf
> > 
> > So, you should at least set it that high. Maybe a bit higher.
> > 
> > > 2. Another thing I would like to follow up on is the difference between
> > > q_depth and backlog_limit. My assumption was if there is any drop due
> > > to
> > 
> > a
> > 
> > > burst of events it would be addressed by the backlog limit. Just would
> > 
> > like
> > 
> > > some clarification on this and how this is an event dispatcher issue?
> > 
> > The backlog limit is inside the kernel. This is the buffer that holds
> > events
> > that are waiting for the audit daemon to offload them. Once the audit
> > daemon
> > has them, it sends it to the dispatcher which also buffers events because
> > not
> > all plugins are able to receive the events as soon as they arrive at the
> > dispatcher.
> > 
> > So, for brief bursts, the kernel backlog will handle the load. But once
> > they
> > are pulled out of the kernel, the q_depth controls how much to hold
> > waiting
> > for plugins. If this number needs to increase much, then the plugins are
> > having problems. The syslog plugin should be fine. I'd look more at the
> > af_unix plugin. The client that attaches to it needs to unload events
> > quickly. I'd investigate the af_unix client to see if it's the problem.
> > 
> > Cheers,
> > -Steve