USB-hotplugging fails with "failed to load cgroup BPF prog: Operation not permitted" on cgroups v2

[Pol Van Aubel]

Hi all,

I've disabled cgroups v1 on my system with the kernel boot option
"systemd.unified_cgroup_hierarchy=1". Since doing so, USB hotplugging
fails to work, seemingly due to a permissions problem with BPF. Please
note that the technique I'm going to describe worked just fine for
hotplugging USB devices to running domains until this change.
Attaching / detaching USB devices when the domain is down still works as
expected.

I get the same error when attaching a device in virt-manager, as I do
when running the following command:

sudo virsh attach-device wenger /dev/stdin --persistent <<END
<hostdev mode='subsystem' type='usb' managed='yes'>
  <source startupPolicy='optional'>
    <vendor id='0x046d' />
    <product id='0xc215' />
  </source>
</hostdev>
END

This returns
error: Failed to attach device from /dev/stdin
error: failed to load cgroup BPF prog: Operation not permitted


virt-manager returns basically the same error, but for completeness'
sake, here it is:

failed to load cgroup BPF prog: Operation not permitted

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/addhardware.py", line 1327, in _add_device
    self.vm.attach_device(dev)
  File "/usr/share/virt-manager/virtManager/object/domain.py", line 920, in attach_device
    self._backend.attachDevice(devxml)
  File "/usr/lib/python3.8/site-packages/libvirt.py", line 590, in attachDevice
    if ret == -1: raise libvirtError ('virDomainAttachDevice() failed', dom=self)
libvirt.libvirtError: failed to load cgroup BPF prog: Operation not permitted


Now, libvirtd is running as root, so I don't understand why any
operation on BPF programs is not permitted. I've dug into libvirt's code
a bit to see what is throwing this error and it boils down to
<https://github.com/libvirt/libvirt/blob/7d608469621a3fda72dff2a89308e68cc9fb4c9a/src/util/vircgroupv2devices.c#L292-L296>
and
<https://github.com/libvirt/libvirt/blob/02bf7cc68bfc76242f02d23e73cad36618f3f790/src/util/virbpf.c#L54>
but I have no clue what that syscall is doing, so that's where my
debugging capability basically ends.

Maybe this is something as simple as setting the right ACL somewhere. I
haven't touched /etc/libvirt/qemu.conf except for setting nvram. There
*is* something about cgroup_device_acl there but afaict that's for
cgroups v1, when there was still a device cgroup controller. Any help
would be greatly appreciated.


Domain log files:
Upon execution of the above commands, nothing gets added to the domain
log in /var/log/qemu/wenger.log, so I've decided they're likely
irrelevant to the issue. Please ask for any additional info required.


System information:
Arch Linux, (normal) kernel 5.4.11
libvirt 5.10.0
qemu 4.2.0, using KVM.
Host system is x86_64 on an intel 5820k.
Guest system is probably irrelevant, but is Windows 10 on the same.


Possibly relevant kernel build options:
$ zgrep BPF /proc/config.gz                                                       
[22:55:52]: zgrep BPF /proc/config.gz

CONFIG_CGROUP_BPF=y
CONFIG_BPF=y
CONFIG_BPF_SYSCALL=y
CONFIG_BPF_JIT_ALWAYS_ON=y
CONFIG_IPV6_SEG6_BPF=y
CONFIG_NETFILTER_XT_MATCH_BPF=m
# CONFIG_BPFILTER is not set
CONFIG_NET_CLS_BPF=m
CONFIG_NET_ACT_BPF=m
CONFIG_BPF_JIT=y
CONFIG_BPF_STREAM_PARSER=y
CONFIG_LWTUNNEL_BPF=y
CONFIG_HAVE_EBPF_JIT=y
CONFIG_BPF_EVENTS=y
# CONFIG_BPF_KPROBE_OVERRIDE is not set
# CONFIG_TEST_BPF is not set


Regards,

Pol Van Aubel