Races / crashes in shutdown of libvirtd daemon

[Michal Privoznik]

On 11/13/20 7:34 AM, Nikolay Shirokovskiy wrote:
> 
> 
> On 12.11.2020 20:12, Michal Privoznik wrote:
>> On 4/27/20 5:54 PM, Daniel P. Berrangé wrote:
>>> We got a new BZ filed about a libvirtd crash in shutdown
>>>
>>>     https://bugzilla.redhat.com/show_bug.cgi?id=1828207
>>
>> And there is another one:
>>
>> https://bugzilla.redhat.com/show_bug.cgi?id=1895359
>>
>> The problem is that when host is shutting down we get PrepareForShutdown() signal on dbus and spawn a thread that will eventually call qemuStateStop(). But before it gets a chance to run the main thread quits the event loop and calls qemuStateCleanup() freeing the qemu driver. After all this the dbus signal handling thread gets to run only to find qemu_driver=0x0 and thus crash.
>>
>>  From the fact that the event loop quit I deduct that we were sent a SIGTERM which means there was no guest running otherwise we would inhibit the shutdown, so qemuStateStop() would be a NOP anyway. So maybe the fix for this particular case indeed is to check whether qemu_driver == 0x0.
>>
>> Anyway, I think your idea is sound.
>>
> 
> 
> Hi, Michal.
> 
> I added code to handle shutdown as Dan proposed in [1]. It handles most busy threads which are rpc and worker and per-VM event loop
> threads in qemu driver. But there are still other drivers and short living threads here and there [2]. So thread that spawns
> daemonStop is on of these untamed threads.
> 
> So we need to join this thread in daemonShutdownWait.

Ah, so are you saying that my patch is not correct?

https://www.redhat.com/archives/libvir-list/2020-November/thread.html

and I see you replied, so let's continue discussion there.

Michal