<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv="Content-Type" content="text/html; charset=us-ascii"> <meta name="Generator" content="Microsoft Word 15 (filtered medium)"> <style></style> </head> <body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word"> <div class="WordSection1"> <p class="MsoNormal">A little more testing shows this happens if I try to do anything with the VM when it is paused after a checkpoint. And the first checkpoint taken AFTER a checkpoint restore always leaves the VM paused for > 5 minutes. Weird. <o:p></o:p></p> <p class="MsoNormal"><o:p> </o:p></p> <div> <div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in"> <p class="MsoNormal" style="margin-left:.5in"><b>From:</b> Leek, Jim <leek2@llnl.gov> <br> <b>Sent:</b> Wednesday, September 8, 2021 5:31 PM<br> <b>To:</b> Tony Brian Albers <tba@kb.dk>; virt-tools-list@redhat.com<br> <b>Subject:</b> Re: Triggering a checkpoint from inside the VM<o:p></o:p></p> </div> </div> <p class="MsoNormal" style="margin-left:.5in"><o:p> </o:p></p> <div> <p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;color:black">Here's the error I sometimes get on checkpoint/restore. It just happened again. In this test I had at least a 40 second break between checkpoint and restore, so extra sleep doesn't seem to help much.<o:p></o:p></span></p> </div> <div> <p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p> </div> <div> <p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;color:black">Error unpausing domain: Timed out during operation: cannot acquire state change lock (held by monitor=qemuDispatchDomainMonitorCommand) <o:p></o:p></span></p> <div> <p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p> </div> <div> <p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;color:black">Traceback (most recent call last):<o:p></o:p></span></p> </div> <div> <p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;color:black"> File "/usr/share/virt-manager/virtManager/asyncjob.py", line 75, in cb_wrapper<o:p></o:p></span></p> </div> <div> <p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;color:black"> callback(asyncjob, *args, **kwargs)<o:p></o:p></span></p> </div> <div> <p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;color:black"> File "/usr/share/virt-manager/virtManager/asyncjob.py", line 111, in tmpcb<o:p></o:p></span></p> </div> <div> <p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;color:black"> callback(*args, **kwargs)<o:p></o:p></span></p> </div> <div> <p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;color:black"> File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 66, in newfn<o:p></o:p></span></p> </div> <div> <p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;color:black"> ret = fn(self, *args, **kwargs)<o:p></o:p></span></p> </div> <div> <p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;color:black"> File "/usr/share/virt-manager/virtManager/object/domain.py", line 1312, in resume<o:p></o:p></span></p> </div> <div> <p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;color:black"> self._backend.resume()<o:p></o:p></span></p> </div> <div> <p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;color:black"> File "/usr/lib64/python3.6/site-packages/libvirt.py", line 2174, in resume<o:p></o:p></span></p> </div> <div> <p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;color:black"> if ret == -1: raise libvirtError ('virDomainResume() failed', dom=self)<o:p></o:p></span></p> </div> <p class="MsoNormal" style="margin-left:.5in"><span style="font-size:12.0pt;color:black">libvirt.libvirtError: Timed out during operation: cannot acquire state change lock (held by monitor=qemuDispatchDomainMonitorCommand)<o:p></o:p></span></p> </div> <div class="MsoNormal" align="center" style="margin-left:.5in;text-align:center"> <hr size="2" width="98%" align="center"> </div> <div id="divRplyFwdMsg"> <p class="MsoNormal" style="margin-left:.5in"><b><span style="color:black">From:</span></b><span style="color:black"> Tony Brian Albers <<a href="mailto:tba@kb.dk">tba@kb.dk</a>><br> <b>Sent:</b> Wednesday, September 8, 2021 12:06 AM<br> <b>To:</b> Leek, Jim <<a href="mailto:leek2@llnl.gov">leek2@llnl.gov</a>>; <a href="mailto:virt-tools-list@redhat.com"> virt-tools-list@redhat.com</a> <<a href="mailto:virt-tools-list@redhat.com">virt-tools-list@redhat.com</a>><br> <b>Subject:</b> Re: Triggering a checkpoint from inside the VM</span> <o:p></o:p></p> <div> <p class="MsoNormal" style="margin-left:.5in"> <o:p></o:p></p> </div> </div> <div> <div> <p class="MsoNormal" style="margin-left:.5in">On 08/09/2021 08.31, Leek, Jim wrote:<br> > 1. I guess you mean by Using a shared disk? That could work, I had some trouble with the shared disk before, I don’t think I ever got it to work. I’ll have to check my notes. IT has the host machine locked down very securely in some ways, and sometimes that causes trouble.<br> <br> On my host I've set up a local network that only the VM's and the host <br> has access to. That makes it possible to do smth like:<br> <br> user@host$ ssh user@vm ls -l somefile<br> <br> and then compare that output to something you've defined, and trigger <br> the savevm based on that.<br> <br> > <br> > 2. I did specify the settings in virt-manager and I can use virsh. But sometimes I get an error from libvirt when doing a series of savevm and loadvm in a row. I have a hope that using qemu-kvm without libvirt might make it more reliable. But as I said, I’m having trouble getting the networking and display working properly without libvirt.<br> > <br> <br> I think libvirt is necessary to make networking and display work(not <br> entirely sure). But the errors you get might be because the processes <br> are scheduled too close to one another, see if you can put something <br> like a sleep() in between them. I think 10 seconds will do.<br> <br> /tony<br> <br> > <br> > ---<br> > Sent from Workspace ONE Boxer<<a href="https://urldefense.us/v3/__https:/whatisworkspaceone.com/boxer__;!!G2kpM7uM-TzIFchu!hbhAdjLTl6sGjvB4cfYJDpIObK0BhNLmWmmi0NEBWX7ls4h84l2slu6DAIhiKh0$%20">https://urldefense.us/v3/__https://whatisworkspaceone.com/boxer__;!!G2kpM7uM-TzIFchu!hbhAdjLTl6sGjvB4cfYJDpIObK0BhNLmWmmi0NEBWX7ls4h84l2slu6DAIhiKh0$ </a>><br> > <br> > On September 7, 2021 at 10:58:03 PM PDT, Tony Brian Albers <<a href="mailto:tba@kb.dk">tba@kb.dk</a>> wrote:<br> > On 08/09/2021 06.22, Leek, Jim wrote:<br> > <br> >><br> >> 1. Connect to the qemu monitor with telnet from inside the VM. (Therefore skipping the whole ssh remote command thing.)<br> > <br> > I'd go the other way around, make the guest touch a file somewhere and<br> > let the host check if the file is there/has been updated, and based on<br> > that, run the savevm process.<br> > <br> > <br> >> 2. Run the VM without virt-manager (perhaps that would be simpler?)<br> >><br> > <br> > I think you can specify the settings for the VM through virt-manager and<br> > then just use virsh to manage it afterwards.<br> > <br> > Also, check out github for kvm backup scripts, I know there are some<br> > that you could use.<br> > <br> > <br> > HTH<br> > <br> > /tony<br> > <br> > <br> > --<br> > Tony Albers - Systems Architect - Data Department, Royal Danish Library,<br> > Victor Albecks Vej 1, 8000 Aarhus C, Denmark<br> > Tel: +45 2566 2383 - CVR/SE: 2898 8842 - EAN: 5798000792142<br> > <br> <br> <br> -- <br> Tony Albers - Systems Architect - Data Department, Royal Danish Library, <br> Victor Albecks Vej 1, 8000 Aarhus C, Denmark<br> Tel: +45 2566 2383 - CVR/SE: 2898 8842 - EAN: 5798000792142<o:p></o:p></p> </div> </div> </div> </body> </html>