[PATCH] audit: improve robustness of the audit queue handling

[Paul Moore]

On Mon, Dec 13, 2021 at 1:31 PM Paul Moore <paul at paul-moore.com> wrote:
>
> If the audit daemon were ever to get stuck in a stopped state the
> kernel's kauditd_thread() could get blocked attempting to send audit
> records to the userspace audit daemon.  With the kernel thread
> blocked it is possible that the audit queue could grow unbounded as
> certain audit record generating events must be exempt from the queue
> limits else the system enter a deadlock state.
>
> This patch resolves this problem by lowering the kernel thread's
> socket sending timeout from MAX_SCHEDULE_TIMEOUT to HZ/10 and tweaks
> the kauditd_send_queue() function to better manage the various audit
> queues when connection problems occur between the kernel and the
> audit daemon.  With this patch, the backlog may temporarily grow
> beyond the defined limits when the audit daemon is stopped and the
> system is under heavy audit pressure, but kauditd_thread() will
> continue to make progress and drain the queues as it would for other
> connection problems.  For example, with the audit daemon put into a
> stopped state and the system configured to audit every syscall it
> was still possible to shutdown the system without a kernel panic,
> deadlock, etc.; granted, the system was slow to shutdown but that is
> to be expected given the extreme pressure of recording every syscall.
>
> The timeout value of HZ/10 was chosen primarily through
> experimentation and this developer's "gut feeling".  There is likely
> no one perfect value, but as this scenario is limited in scope (root
> privileges would be needed to send SIGSTOP to the audit daemon), it
> is likely not worth exposing this as a tunable at present.  This can
> always be done at a later date if it proves necessary.
>
> Cc: stable at vger.kernel.org
> Fixes: 5b52330bbfe63 ("audit: fix auditd/kernel connection state tracking")
> Reported-by: Gaosheng Cui <cuigaosheng1 at huawei.com>
> Signed-off-by: Paul Moore <paul at paul-moore.com>
> ---
>  kernel/audit.c |   21 ++++++++++-----------
>  1 file changed, 10 insertions(+), 11 deletions(-)

Merged into audit/stable-5.16, thanks for the help everyone.  Assuming
all of the automated testing goes well, I'll plan on sending this up
to Linus tomorrow.

-- 
paul moore
www.paul-moore.com