Libvirtd Fails to Launch First Time

[Laine Stump]

On 8/8/20 10:44 AM, Ken Swenson wrote:
> Hello,
> 
> I'm having a quite odd issue with libvirtd. I have it set to start on 
> boot via systemd service, however it seems to fail and the service 
> 'succeeds' and does not continue running the daemon. There is nothing in 
> the journal logs however the libvirtd debug logs seem to show
> 
>     2020-08-08 13:55:26.362+0000: 1386: debug : virCommandRunAsync:2619
>     : About to run ip link set lo netns -1
>     2020-08-08 13:55:26.362+0000: 1386: debug : virFileClose:134 :
>     Closed fd 26
>     2020-08-08 13:55:26.362+0000: 1386: debug : virFileClose:134 :
>     Closed fd 28
>     2020-08-08 13:55:26.362+0000: 1386: debug : virFileClose:134 :
>     Closed fd 30
>     2020-08-08 13:55:26.362+0000: 1386: debug : virCommandRunAsync:2621
>     : Command result 0, with PID 1610
>     2020-08-08 13:55:26.390+0000: 1386: debug : virCommandRun:2461 :
>     Result fatal signal 2, stdout: '' stderr: 'RTNETLINK answers: No
>     such process
> 
> I am not sure if this is relevant to the problem or not as I do not see 
> anything else in the logs that indicates an issue during the launch 
> process, other than many file descriptors closing but I assume that is 
> normal. If I then start libvirtd a second time after it has been 
> launched once it runs fine. I don't believe it is a race condition on 
> boot as if I disable the auto start and manually start it it will still 
> fail in the same way the first time.
> 
> If anyone has any ideas on what I may be able to check to fix this I 
> would really appreciate it. It has been bugging me for some time now.

The command "ip link set lo netns -1" is called by the LXC driver 
(during its driver init) as a check to see if the OS supports network 
namespaces. This is done in the function lxcCheckNetNsSupport(), which 
was added to libvirt's LXC driver in commit 99ded85f4 in 2008, has 0 
comments about how it works, and has been functionally unchanged since then.

It appears that on your system for whatever reason the attempt to send 
the netlink message that sets the netns for lo to -1 fails the first 
time it is run, causing lxcCheckNetNsSupport() to return false. I don't 
see how this would cause libvirtd to exit though (I *guess* that's what 
you're saying happens?). I'm guessing this is a red herring.


What is the practical problem you have? After this "success" are you not 
able to start guests or run commands with virsh? Or were you just 
surprised that the libvirtd process wasn't there when you looked? If the 
latter, that could be due to using socket activation - libvirtd is no 
longer running all the time; systemd sets up the listening sockets and 
starts libvirtd when needed; if 120 seconds go by and there are no 
guests running and no management clients connected to libvirtd, then it 
will automatically exit.



(BTW, I'm probably the least knowledgeable person about kernel 
namespaces around, but I thought that all Linux systems have had 
namespace support for a very long time. Does anybody ever actually 
disable it? Do we really need this check?)