Libvirtd Fails to Launch First Time

[Ken Swenson]

Indeed it seems to have been a red herring... Your comment about the
systemd socket gave be the idea to ensure it was enabled and sure enough
the libvirtd.service was enabled and the libvirtd.socket was not, so
libvirtd was exiting without activity and without a socket so I was
unable to start guests or connect at all. Enabling the socket seems to
have fixed all my issues. Sometimes issues are surface level when I'm
already digging 10ft under. Thanks!

On 8/9/20 12:11 AM, Laine Stump wrote:
> On 8/8/20 10:44 AM, Ken Swenson wrote:
>> Hello,
>>
>> I'm having a quite odd issue with libvirtd. I have it set to start on
>> boot via systemd service, however it seems to fail and the service
>> 'succeeds' and does not continue running the daemon. There is nothing
>> in the journal logs however the libvirtd debug logs seem to show
>>
>>     2020-08-08 13:55:26.362+0000: 1386: debug : virCommandRunAsync:2619
>>     : About to run ip link set lo netns -1
>>     2020-08-08 13:55:26.362+0000: 1386: debug : virFileClose:134 :
>>     Closed fd 26
>>     2020-08-08 13:55:26.362+0000: 1386: debug : virFileClose:134 :
>>     Closed fd 28
>>     2020-08-08 13:55:26.362+0000: 1386: debug : virFileClose:134 :
>>     Closed fd 30
>>     2020-08-08 13:55:26.362+0000: 1386: debug : virCommandRunAsync:2621
>>     : Command result 0, with PID 1610
>>     2020-08-08 13:55:26.390+0000: 1386: debug : virCommandRun:2461 :
>>     Result fatal signal 2, stdout: '' stderr: 'RTNETLINK answers: No
>>     such process
>>
>> I am not sure if this is relevant to the problem or not as I do not
>> see anything else in the logs that indicates an issue during the
>> launch process, other than many file descriptors closing but I assume
>> that is normal. If I then start libvirtd a second time after it has
>> been launched once it runs fine. I don't believe it is a race
>> condition on boot as if I disable the auto start and manually start
>> it it will still fail in the same way the first time.
>>
>> If anyone has any ideas on what I may be able to check to fix this I
>> would really appreciate it. It has been bugging me for some time now.
>
> The command "ip link set lo netns -1" is called by the LXC driver
> (during its driver init) as a check to see if the OS supports network
> namespaces. This is done in the function lxcCheckNetNsSupport(), which
> was added to libvirt's LXC driver in commit 99ded85f4 in 2008, has 0
> comments about how it works, and has been functionally unchanged since
> then.
>
> It appears that on your system for whatever reason the attempt to send
> the netlink message that sets the netns for lo to -1 fails the first
> time it is run, causing lxcCheckNetNsSupport() to return false. I
> don't see how this would cause libvirtd to exit though (I *guess*
> that's what you're saying happens?). I'm guessing this is a red herring.
>
>
> What is the practical problem you have? After this "success" are you
> not able to start guests or run commands with virsh? Or were you just
> surprised that the libvirtd process wasn't there when you looked? If
> the latter, that could be due to using socket activation - libvirtd is
> no longer running all the time; systemd sets up the listening sockets
> and starts libvirtd when needed; if 120 seconds go by and there are no
> guests running and no management clients connected to libvirtd, then
> it will automatically exit.
>
>
>
> (BTW, I'm probably the least knowledgeable person about kernel
> namespaces around, but I thought that all Linux systems have had
> namespace support for a very long time. Does anybody ever actually
> disable it? Do we really need this check?)
>