[libvirt-users] libvirt unavailable while a VM is in migration?]

[Igor Serebryany]

I've opened a bug to track this issue:
https://bugzilla.redhat.com/show_bug.cgi?id=666266

I have tried to work around the issue by making my migrations non-live,
hoping that the shorter downtime will make this less of a big deal. But
migrating busy domains with lots of RAM still takes a surprisingly long
time even if the domain is paused during migration -- something I did
not expect.

For instance, a domain with 8 GB of ram migrating over a gigabit network
takes about 20 seconds if the domain is idle and not using much ram.
However, if I start a run of 'stress' inside the VM which uses 6 gigs of
RAM in random malloc/free calls, the same domain might take 20 minutes
to migrate. 

Has anyone else experienced this? Is there some kind of compression
going on when doing tunneled migrations which breaks down when the pages
are dirty?

--Igor

On Thu, Dec 23, 2010 at 06:13:35AM +1100, Justin Clift wrote:
> On 23/12/2010, at 5:44 AM, Igor Serebryany wrote:
> > On Mon, Dec 20, 2010 at 02:14:42PM +0100, Jiri Denemark wrote:
> >> Ouch, I wonder if that could be the reason...
> > 
> > Just to make sure that my strange compiling/packages aren't causing the
> > problem, I've set up two servers running fedora 14 with the official
> > libvirt packages. i experience the same problem -- while a vm is in
> > migration, nothing else works. my traces reveal libvirt getting stuck at
> > the same exact spot as on my debian machines.
> 
> This sounded familiar, so went looking in git history.  I *think* this is
> what was sounding familiar, from ~ libvirt 0.8.2:
> 
> commit 755b53f946107539ba6faf3022450b3d0bbe1d78
> Author: Daniel P. Berrange <berrange at redhat.com>
> Date:   Thu Jun 24 19:15:27 2010 +0100
> 
>     Avoid blocking all APIs during incoming migration
>    
>     During incoming migration the QEMU monitor is not able to be
>     used. The incoming migration code did not keep hold of the
>     job lock because migration is split across multiple API calls.
>     This meant that further monitor commands on the guest would
>     hang until migration finished with no timeout.
>    
>     In this change the qemuDomainMigratePrepare method sets the
>     job flag just before it returns. The qemuDomainMigrateFinish
>     method checks for this job flag & clears it once done. This
>     prevents any use of the monitor between prepare+finish steps.
>    
>     The qemuDomainGetJobInfo method is also updated to refresh
>     the job elapsed time. This means that virsh domjobinfo can
>     return time data during incoming migration
>    
>     * src/qemu/qemu_driver.c: Keep a job active during incoming
>       migration. Refresh job elapsed time when returning job info
> 
> Anyone know if this is actually relevant?
> 
> 
> > i am desperate to get this bug fixed, and i don't think i have the
> > resources to try to patch it myself. is there anything i can do to help
> > you guys? more debugging info? foot massage?
> 
> Good idea.  Foot massages would definitely be an incentive I
> could be convinced with. :)
> 
> (it would probably help if I was the right kind of coder for this,
> but I'm not)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://listman.redhat.com/archives/libvirt-users/attachments/20101229/a60c4429/attachment.sig>