Running libvirtd inside chroot (mock to be precise)

[Marcin Sobczyk]

Hi,

I was wondering whether it's possible to run libvirtd inside a chroot 
environment.
The assumption is that only one instance of libvirtd would be running on 
the machine at a time, but still, inside chroot.

Currently in my chroot env I have:

- /dev/kvm added with mknod
- /dev/vhost-net added with mknod
- mounted:
   - /dev/net
   - /dev/shm
   - /run/dbus

When I run libvirtd in non-daemon mode I get one error:

2020-05-13 10:06:27.480+0000: 28555: error : virNetDevBridgeCreate:484 : 
error creating bridge interface virbr0: File exists

but it seems to me that libvirtd can live with that.

When I try to run a VM however I get a qemu error reported:

2020-05-13 08:56:51.643+0000: starting up libvirt version: 4.5.0, 
package: 35.3.module_el8.1.0+297+df420408 (CentOS Buildsys 
<bugs at centos.org>, 2020-04-10-11:06:38, ), qemu version: 
2.12.0qemu-kvm-2.12.0-88.module_el8.1.0+297+df420408.3, kernel: 
4.18.0-80.el8.x86_64, hostname: .....
LC_ALL=C 
PATH=/var/lib/ci_toolbox:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin 
HOME=/root/ovirt-system-tests USER=root LOGNAME=root QEMU_AUDIO_DRV=none 
/usr/libexec/qemu-kvm -name 
guest=992415aa-lago-basic-suite-master-engine,debug-threads=on -S 
-object 
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-992415aa-lago-basic-/master-key.aes 
-machine 
pc-i440fx-rhel7.6.0,accel=kvm,usb=off,vmport=off,dump-guest-core=off 
-cpu host -m 4096 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 
-object iothread,id=iothread1 -numa node,nodeid=0,cpus=0,mem=2048 -numa 
node,nodeid=1,cpus=1,mem=2048 -uuid 10ed4b42-bb14-4b03-a411-c62fb1181c2d 
-display none -no-user-config -nodefaults -chardev 
socket,id=charmonitor,fd=31,server,nowait -mon 
chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown 
-global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot 
menu=off,strict=on -device 
virtio-scsi-pci,iothread=iothread1,id=scsi0,num_queues=2,bus=pci.0,addr=0x5 
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive 
file=/root/ovirt-system-tests/deployment-basic-suite-master/default/images/lago-basic-suite-master-engine_root.qcow2,format=qcow2,if=none,id=drive-virtio-disk0,serial=1,cache=writeback,discard=unmap 
-device 
virtio-blk-pci,iothread=iothread1,scsi=off,num-queues=1,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on 
-drive 
file=/root/ovirt-system-tests/deployment-basic-suite-master/default/images/lago-basic-suite-master-engine_nfs.raw,format=raw,if=none,id=drive-scsi0-0-0-0,serial=2,cache=writeback,discard=unmap 
-device 
scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,write-cache=on 
-drive 
file=/root/ovirt-system-tests/deployment-basic-suite-master/default/images/lago-basic-suite-master-engine_iscsi.raw,format=raw,if=none,id=drive-scsi0-0-0-2,serial=3,cache=writeback,discard=unmap 
-device 
scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=2,drive=drive-scsi0-0-0-2,id=scsi0-0-0-2,write-cache=on 
-netdev tap,fds=33:34,id=hostnet0,vhost=on,vhostfds=35:36 -device 
virtio-net-pci,mq=on,vectors=6,netdev=hostnet0,id=net0,mac=54:52:c0:a8:c8:02,bus=pci.0,addr=0x3 
-netdev tap,fds=37:38,id=hostnet1,vhost=on,vhostfds=39:40 -device 
virtio-net-pci,mq=on,vectors=6,netdev=hostnet1,id=net1,mac=54:52:c0:a8:ca:02,bus=pci.0,addr=0x4 
-chardev socket,id=charchannel0,fd=41,server,nowait -device 
virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 
-chardev pty,id=charconsole0 -device 
virtconsole,chardev=charconsole0,id=console0 -device 
cirrus-vga,id=video0,bus=pci.0,addr=0x2 -object 
rng-random,id=objrng0,filename=/dev/urandom -device 
virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x9 -sandbox 
on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny 
-msg timestamp=on
2020-05-13 08:56:51.643+0000: Domain id=1 is tainted: high-privileges
2020-05-13 08:56:51.643+0000: Domain id=1 is tainted: host-cpu
libvirt: QEMU Driver error : internal error: Unable to find any /dev mount
2020-05-13 08:56:51.654+0000: shutting down, reason=failed

And this one currently is blocking me.
Did anyone try such a setup?
Any help would be appreciated.

Regards, Marcin