USB-hotplugging fails with "failed to load cgroup BPF prog: Operation not permitted" on cgroups v2

[Pavel Hrdina]

On Sat, Jan 18, 2020 at 11:17:11PM +0100, Pol Van Aubel wrote:
> Hi all,
> 
> I've disabled cgroups v1 on my system with the kernel boot option
> "systemd.unified_cgroup_hierarchy=1". Since doing so, USB hotplugging
> fails to work, seemingly due to a permissions problem with BPF. Please
> note that the technique I'm going to describe worked just fine for
> hotplugging USB devices to running domains until this change.
> Attaching / detaching USB devices when the domain is down still works as
> expected.
> 
> I get the same error when attaching a device in virt-manager, as I do
> when running the following command:
> 
> sudo virsh attach-device wenger /dev/stdin --persistent <<END
> <hostdev mode='subsystem' type='usb' managed='yes'>
>   <source startupPolicy='optional'>
>     <vendor id='0x046d' />
>     <product id='0xc215' />
>   </source>
> </hostdev>
> END
> 
> This returns
> error: Failed to attach device from /dev/stdin
> error: failed to load cgroup BPF prog: Operation not permitted
> 
> 
> virt-manager returns basically the same error, but for completeness'
> sake, here it is:
> 
> failed to load cgroup BPF prog: Operation not permitted
> 
> Traceback (most recent call last):
>   File "/usr/share/virt-manager/virtManager/addhardware.py", line 1327, in _add_device
>     self.vm.attach_device(dev)
>   File "/usr/share/virt-manager/virtManager/object/domain.py", line 920, in attach_device
>     self._backend.attachDevice(devxml)
>   File "/usr/lib/python3.8/site-packages/libvirt.py", line 590, in attachDevice
>     if ret == -1: raise libvirtError ('virDomainAttachDevice() failed', dom=self)
> libvirt.libvirtError: failed to load cgroup BPF prog: Operation not permitted
> 
> 
> Now, libvirtd is running as root, so I don't understand why any
> operation on BPF programs is not permitted. I've dug into libvirt's code
> a bit to see what is throwing this error and it boils down to
> <https://github.com/libvirt/libvirt/blob/7d608469621a3fda72dff2a89308e68cc9fb4c9a/src/util/vircgroupv2devices.c#L292-L296>
> and
> <https://github.com/libvirt/libvirt/blob/02bf7cc68bfc76242f02d23e73cad36618f3f790/src/util/virbpf.c#L54>
> but I have no clue what that syscall is doing, so that's where my
> debugging capability basically ends.
> 
> Maybe this is something as simple as setting the right ACL somewhere. I
> haven't touched /etc/libvirt/qemu.conf except for setting nvram. There
> *is* something about cgroup_device_acl there but afaict that's for
> cgroups v1, when there was still a device cgroup controller. Any help
> would be greatly appreciated.
> 
> 
> Domain log files:
> Upon execution of the above commands, nothing gets added to the domain
> log in /var/log/qemu/wenger.log, so I've decided they're likely
> irrelevant to the issue. Please ask for any additional info required.
> 
> 
> System information:
> Arch Linux, (normal) kernel 5.4.11
> libvirt 5.10.0
> qemu 4.2.0, using KVM.
> Host system is x86_64 on an intel 5820k.
> Guest system is probably irrelevant, but is Windows 10 on the same.
> 
> 
> Possibly relevant kernel build options:
> $ zgrep BPF /proc/config.gz                                                       
> [22:55:52]: zgrep BPF /proc/config.gz
> 
> CONFIG_CGROUP_BPF=y
> CONFIG_BPF=y
> CONFIG_BPF_SYSCALL=y
> CONFIG_BPF_JIT_ALWAYS_ON=y
> CONFIG_IPV6_SEG6_BPF=y
> CONFIG_NETFILTER_XT_MATCH_BPF=m
> # CONFIG_BPFILTER is not set
> CONFIG_NET_CLS_BPF=m
> CONFIG_NET_ACT_BPF=m
> CONFIG_BPF_JIT=y
> CONFIG_BPF_STREAM_PARSER=y
> CONFIG_LWTUNNEL_BPF=y
> CONFIG_HAVE_EBPF_JIT=y
> CONFIG_BPF_EVENTS=y
> # CONFIG_BPF_KPROBE_OVERRIDE is not set
> # CONFIG_TEST_BPF is not set

Hi

I've installed clean archlinux to try this out and it works as expected,
I'm able to attach USB device into a VM.

My system env is mostly the same as yours except for kernel version:

    kernel 5.4.13
    libvirt 5.10.0
    qemu 4.2.0, using KVM.

Please enable libvirt debug logs [1] and share the output with us.

Pavel

[1] <https://wiki.libvirt.org/page/DebugLogs>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/libvirt-users/attachments/20200120/9b1b1f43/attachment.sig>