virtlogd spinning on 100% CPU with the latest libvirt
Ján Tomko
jtomko at redhat.com
Wed Feb 19 00:34:26 UTC 2020
On Tue, Feb 18, 2020 at 08:34:53PM +0100, Andrea Bolognani wrote:
>[Dropped Peter from CC. Please don't CC individual developers
> unless they've explicitly requested that you do so.]
>
>On Mon, 2020-02-17 at 17:50 +0000, Richard W.M. Jones wrote:
>> Build libvirt from git (ccf7567329f).
>>
>> Using the libvirt ‘run’ script, run something like
>> libguestfs-test-tool. I think basically any command which runs a
>> guest will do. NB These commands are all run as NON-root:
>>
>> killall libvirtd lt-libvirtd virtlogd lt-virtlogd
>> ./build/run libguestfs-test-tool
>>
>> Now there will be a lt-virtlogd process using 100% of CPU:
>>
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 2572972 rjones 20 0 47880 16256 14516 R 100.0 0.1 0:19.27 lt-virt+
>
>It's actually worse than that: not only virtlogd usese an
>unwarranted amount of CPU, but it also keeps the log file for the
>domain busy, thus preventing the same domain from being started
>again:
>
> $ virsh start alpine
> Domain alpine started
>
> $ virsh destroy alpine
> Domain alpine destroyed
>
> $ virsh start alpine
> error: Failed to start domain alpine
> error: can't connect to virtlogd: Cannot open log file: '/var/log/libvirt/qemu/alpine.log': Device or resource busy
>
> $ sudo lsof | grep alpine.log
> virtlogd 146845 root 16w REG 253,0 35103 17195654 /var/log/libvirt/qemu/alpine.log
> $
>
>Restarting virtlogd makes thing operational again:
>
> $ sudo systemctl restart virtlogd
> $ virsh start alpine
> Domain alpine started
>
> $
>
>My guess is that virtlogd doesn't realize the QEMU process is gone,
>and sits there spinning forever waiting for some output that will
>never arrive.
Yeah, since the switch to GLib event loop, the virLogHandlerDomainLogFileEvent
which was supposed to clean that up is no longer called on hangup.
My naive fix:
https://www.redhat.com/archives/libvir-list/2020-February/msg00651.html
Jano
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20200219/84d5b5b3/attachment-0001.sig>
More information about the libvir-list
mailing list