[RFC] audit support for BPF notification

[Jiri Olsa]

On Mon, Aug 12, 2019 at 09:49:43AM -0400, Steve Grubb wrote:
> On Monday, August 12, 2019 3:59:22 AM EDT Jiri Olsa wrote:
> > On Fri, Aug 09, 2019 at 01:45:21PM -0400, Steve Grubb wrote:
> > > Hello,
> > > 
> > > On Friday, August 9, 2019 10:18:31 AM EDT Jiri Olsa wrote:
> > > > I posted initial change that allows auditd to log BPF program
> > > > 
> > > > load/unload events, it's in here:
> > > >   https://github.com/linux-audit/audit-userspace/pull/104
> > > 
> > > Thanks for the patch...but we probably should have talked a bit more
> > > before undertaking this effort. We normally do not audit from user space
> > > what happens in the kernel. Doing this can be racy and it keeps auditd
> > > from doing the one job it has - which is to grab events and record them
> > > to disk and send them out the realtime interface.
> > > 
> > > > We tried to push pure AUDIT interface for BPF program notification,
> > > > 
> > > > but it was denied, the discussion is in here:
> > > >   https://marc.info/?t=153866123200003&r=1&w=2
> > > 
> > > Hmm. The email I remember was here:
> > > https://www.redhat.com/archives/linux-audit/2018-October/msg00053.html
> > > 
> > > and was only 2 emails long with no answer to my question. :-)
> > 
> > oops, sry about that, your question was:
> > 	> I'm not sure exactly what the issue is. You can audit for specific
> > 	> syscall
> > 	> and argument. So, if you want to see loads, then you can make a rule
> > 	> like:
> > 	> 
> > 	> -a always,exit -F arch=b64 -S bpf -F a0=5
> > 
> > The problem with above for us is that we also:
> > 
> >   - need to log also other properties of the BPF program,
> >     which are not visible from BPF syscall arguments, like
> >     its ID, JIT status 
> 
> The way this is normally done is to add a supplemental record. For example, 
> when auditing the open syscall, we also get CWD & PATH supplemental records. 
> When auditing connect, we get a SOCKADDR supplemental record. We have 
> requirements around selective audit whereby the admin is in control of what 
> is selected for audit via audit rules. So, what one could do is set a rule 
> for the bpf syscall and then when it triggers, we get these other records 
> added to the bpf syscall event.

right, that was the initial plan, but BPF guys wanted just
single notification system without specific hooks for audit,
so we ended up with perf specific interface

> >     or license info
> 
> This ^^ is not a security issue.
> 
> 
> >   - need to see BPF program UNLOAD, which is not done
> >     via syscall, so those would be unvisible at all
> 
> Is there a place in the kernel where this happens? I could see abnormal 
> termination being something we might want. Does the program go through 
> something like an exit syscall internally?

it's happening in here (kernel/bpf/syscall.c):

	bpf_prog_put
	  __bpf_prog_put
	  {
		    if (atomic_dec_and_test(&prog->aux->refcnt)) {
			perf_event_bpf_event(prog, PERF_BPF_EVENT_PROG_UNLOAD, 0);
			...
	  }

BPF program is released when it drops the reference count,
which is normally when its file descriptor is closed.

However it might get pinned and stay alive even when the initial
file descriptor is closed.. and then there's the networking world,
which might have some other specific ways.. but it all ends up
in bpf_prog_put and zero reference count.

> > > > The outcome of the discussion was to use perf event interface
> > > > for BPF notification and use it in some deamon.. audit was our
> > > > first choice.
> > > > 
> > > > thoughts?
> > > 
> > > I'd like to understand what the basic problem is that needs to be solved.
> > 
> > we need a way for administrators to see the history of loaded BPF
> > programs, to help investigating issues related to BPF.. and the
> > only BPF interface for this data is through perf ring buffer
> 
> That is really not the audit way. Let's keep talking to see where this ends 
> up.

Would you see some other auditing daemon/app in place for this kind of data?

thanks,
jirka