[libvirt] strange stale qemu processes after domain shutdown

[Michal Privoznik]

On 23.10.2015 00:40, Vasiliy Tolstov wrote:
> 2015-10-22 17:38 GMT+03:00 Michal Privoznik <mprivozn at redhat.com>:
>> These are probably left over after daemon restart. I mean, when deamon
>> restarts, it reloads the state XML. However, if there's something wrong
>> (e.g. unknown device), the state XML is ignored and libvirt thinks the
>> domain is shut off, leaving qemu process behind. The unknown device can
>> happen if you run older daemon than the domain has been started with.
>>
> 
> Daemon does not restarts, also to note - domain started without
> define. Libvirt version does not changed....
> 
>> You can check daemon debug logs and see how it loads domain state XMLs.
>> If there's an error somewhere it's possibly this case.
>>
> 
> If domain is not defined, if i'm enable debug in libvirt and restart
> it, this is not helps because qemu process already stalled and libvirt
> does not know about this domain because they not defined.

That does not matter. We keep state XML for all running domains,
regardless if they are persistent or transient. But since the daemon
does not restart, I suspect we have a bug somewhere.
BTW: you can check if the state XML for domain still exists. We pass
-name $domname to qemu, and the state XML should then be:

/var/run/libvirt/qemu/$domname.xml

Also, there is PID of the qemu process - can you check if they match?

> 
>> Of course, the other option would be that we have a bug somewhere where
>> we leave qemu process behind. But the former is more likely than the latter.
> 
> 
> I'm try to add to monitoring (difference with virsh --state-running
> --name vs ps auxww | grep qemu-system | grep -v grep | wc -l) and
> investigate this issue...
> 

Michak