[libvirt-users] Running libvirtd outside of a shell results in qemu defunct processes

[Roman Mohr]

On Tue, Sep 11, 2018 at 2:00 PM Roman Mohr <rmohr at redhat.com> wrote:
>
> Hi,
>
> In KubeVirt we are right now starting libvirt via a wrapper shell
> script. I want to remove that wrapper and directly start libvirtd from
> our golang code. This means that our golang code is PID 1 in the
> container and we start libvirtd as a fork from there.
>
> When I did this, strange things started to happen. Libvirt was still
> able to start and manage qemu processes, but stopping the qemu
> processes stopped working. I can see that all qemu processes, the ones
> with VMs and the ones used by libvirt to do some host capability
> detection, stay in the process table as zombies:
>
> qemu        54  0.0  0.0      0     0 ?        Zs   11:44   0:00
> [qemu-system-i38] <defunct>
> qemu        56  0.2  0.0      0     0 ?        Z    11:44   0:00
> [qemu-system-i38] <defunct>
> qemu        61  0.0  0.0      0     0 ?        Zs   11:44   0:00
> [qemu-system-i38] <defunct>
> qemu        63  0.1  0.0      0     0 ?        Z    11:44   0:00
> [qemu-system-i38] <defunct>
> qemu        70  0.0  0.0      0     0 ?        Zs   11:44   0:00
> [qemu-system-x86] <defunct>
> qemu        72  0.2  0.0      0     0 ?        Z    11:44   0:00
> [qemu-system-x86] <defunct>
> qemu        79  0.0  0.0      0     0 ?        Zs   11:44   0:00
> [qemu-system-x86] <defunct>
> qemu        81  0.2  0.0      0     0 ?        Z    11:44   0:00
> [qemu-system-x86] <defunct>
> qemu        86  0.0  0.0      0     0 ?        Zs   11:44   0:00
> [qemu-system-x86] <defunct>
> qemu        88  0.3  0.0      0     0 ?        Z    11:44   0:00
> [qemu-system-x86] <defunct>
> qemu        93  0.0  0.0      0     0 ?        Zs   11:44   0:00
> [qemu-system-x86] <defunct>
> qemu        95  0.1  0.0      0     0 ?        Z    11:44   0:00
> [qemu-system-x86] <defunct>
>
> So to sum up, when I start our golang wrapper around libvirtd inside a
> bash script without "exec" everything works as usual. If I start in
> the container pid namespace the golang wrapper directly, or use "exec"
> from the bash script, things stop working. I also see the libvirt
> repeatedly tries to kill the qemu processes, but they are stuck.

Never mind. The solution was pretty easy. Since my process is PID 1,
all qemu processes were orphaned by libvirt and reassigned to my
process.
I am now listening for SIGCHILD and reap them by calling waitpid.

Best Regards,
Roman

>
> Maybe someone has a clue why things stop working.
>
> Thanks and Best Regards,
>
> Roman