<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=gb2312">
<style>body { line-height: 1.5; }blockquote { margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em; }body { font-size: 10.5pt; font-family: 'Microsoft YaHei UI'; color: rgb(0, 0, 0); line-height: 1.5; }</style>
</head>
<body>
<div><span></span>Actually,I use the <span style="font-size: 10.5pt; line-height: 1.5; background-color: transparent;">backdoors as you have mentioned</span><span style="font-size: 10.5pt; line-height: 1.5; background-color: transparent;">. I use '</span><span style="font-size: 10.5pt; font-family: 'Microsoft YaHei UI', Tahoma; line-height: normal; background-color: transparent;">virsh
qemu-monitor-command</span><span style="font-size: 10.5pt; line-height: 1.5; background-color: transparent;">' to submit backup jobs and use '</span><span style="font-size: 10.5pt; line-height: 1.5; background-color: transparent;">virsh qemu-monitor-event DOMAIN --timestamp --loop</span><span style="font-size: 10.5pt; line-height: 1.5; background-color: transparent;">'
to monitor the jobs through the QMP protocol. </span></div>
<div><span style="font-size: 10.5pt; line-height: 1.5; background-color: transparent;">for example :</span></div>
<div><span style="font-size: 10.5pt; line-height: 1.5; background-color: transparent;">to do a full backup</span></div>
<div><span microsoftyaheiui?;font-size:14px;color:rgb(0,0,0);background-color:rgba(0,font-weight:normal;font-style:normal;text-decoration:none;?=""><span style="font-family: 'Microsoft YaHei UI', Tahoma; line-height: normal; font-size: 10.5pt; background-color: transparent;">virsh
qemu-monitor-command DOMAIN --pretty </span>{ "execute": "transaction",<br>
"arguments": {<br>
"actions": [<br>
{"type": "block-dirty-bitmap-add",<br>
"data": {"node": "drive0", "name": "bitmap0"} },<br>
{"type": "drive-backup",<br>
"data": {"device": "drive0", "target": "/path/to/full_backup.img",<br>
"sync": "full", "format": "qcow2"} }<br>
]<br>
}<br>
}</span></div>
<div><span microsoftyaheiui?;font-size:14px;color:rgb(0,0,0);background-color:rgba(0,font-weight:normal;font-style:normal;text-decoration:none;?="">refer to https://wiki.qemu.org/Features/IncrementalBackup</span></div>
<div><span style="font-size: 10.5pt; line-height: 1.5; background-color: transparent;"><br>
</span></div>
<div><span style="font-size: 10.5pt; line-height: 1.5; background-color: transparent;">Unfortunately, the minitor process </span><span style="font-size: 10.5pt; line-height: 1.5; background-color: transparent;">'</span><span style="font-size: 10.5pt; line-height: 1.5; background-color: transparent;">virsh qemu-monitor-event DOMAIN --timestamp --loop</span><span style="font-size: 10.5pt; line-height: 1.5; background-color: transparent;">'
was failed to be killed. A few hours later ,there were hundreds of monitor </span><span style="font-size: 10.5pt; line-height: 1.5; background-color: transparent;">processes running(about 200-300), whice caused the 'virsh list' hung. After I have killed the
processes, </span><span style="font-size: 10.5pt; line-height: 1.5; background-color: transparent;"> </span><span style="font-size: 10.5pt; line-height: 1.5; background-color: transparent;">'virsh list' works well. </span></div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<hr style="width: 210px; height: 1px;" color="#b5c4df" size="1" align="left">
<div><span>
<div style="MARGIN: 10px; FONT-FAMILY: verdana; FONT-SIZE: 10pt">
<div>zhanhouliang@outlook.com</div>
</div>
</span></div>
<blockquote style="margin-Top: 0px; margin-Bottom: 0px; margin-Left: 0.5em">
<div> </div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<div style="PADDING-RIGHT: 8px; PADDING-LEFT: 8px; FONT-SIZE: 12px;FONT-FAMILY:tahoma;COLOR:#000000; BACKGROUND: #efefef; PADDING-BOTTOM: 8px; PADDING-TOP: 8px">
<div><b>发件人：</b> <a href="mailto:eblake@redhat.com">Eric Blake</a></div>
<div><b>发送时间：</b> 2019-03-06 23:43</div>
<div><b>收件人：</b> <a href="mailto:zhanhouliang@outlook.com">Zhan Adlun</a>; <a href="mailto:libvirt-users@redhat.com">
libvirt-users@redhat.com</a></div>
<div><b>主题：</b> Re: [libvirt-users] virsh hangs when backup jobs run on the kvm host</div>
</div>
</div>
<div>
<div>On 3/2/19 9:23 PM, Zhan Adlun wrote:</div>
<div>> Hi,</div>
<div>> Has anyone met the same problem that virsh hangs when there are some backup jobs running on the kvm host? I use dirty bitmap supported by qemu to do backups every 6 minitues. About a few hours later, I ran 'virsh list', It hung. Then I stopped the
backup jobs ，nothing was getting better. I had checked the libvirt logs ，but found nothing useful .</div>
<div> </div>
<div>How are you using dirty bitmaps? As my new virDomainBackupBegin() API</div>
<div>has not landed yet, I suspect you are using virQemuMonitorCommand()</div>
<div>(virsh qemu-monitor-command) backdoors, which is explicitly unsupported</div>
<div>(that is, if it breaks, you get to debug it - it exists for development</div>
<div>work, and while 'query-*' QMP commands should be safe, commands that</div>
<div>alter qemu state such as kicking off a backup job may break libvirt</div>
<div>expectations), and which is WHY I'm trying to get incremental backup API</div>
<div>officially supported.</div>
<div> </div>
<div>A precise sequence of commands that you issued in order to reproduce the</div>
<div>virsh hang may be useful in debugging this.</div>
<div> </div>
<div>> Maybe the high I/O caused the problem? or there are connection number limits when using virsh, because I use virsh to check the backup jobs are still running every 10 seconds?</div>
<div>> My os version is ' Ubuntu 12.04 LTS Release: 12.04 Codename: precise,Linux version 4.1.0-generic (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #1 SMP Wed Nov 9 02:04:23 CST 2016'. KVM version is 'QEMU emulator version 2.7.1(Debian 1:2.7+dfsg-3)'.
Hoping for you reply. Thanks!</div>
<div> </div>
<div>Hmm, my incremental backup API will require qemu 4.0; there have been a</div>
<div>number of fixes in the qemu bitmap tracking that have landed since qemu</div>
<div>2.7.1. It could also be that you've hit an actual qemu bug where the QMP</div>
<div>monitor has gone unresponsive.</div>
<div> </div>
<div>-- </div>
<div>Eric Blake, Principal Software Engineer</div>
<div>Red Hat, Inc. +1-919-301-3226</div>
<div>Virtualization: qemu.org | libvirt.org</div>
<div> </div>
</div>
</blockquote>
</body>
</html>