[libvirt-users] Migration problem - takes 5 minutes to start moving the memory

[Marc-Aurèle Brothier - Exoscale]

Hi,

I'm facing a strange issue while doing a migration from an hypervisor to another one. The migration takes for ever to start moving the memory.
The VM had no workload what so ever, just a basic ubuntu image. The versions on the hypervisors are: libvirt 1.2.21, qemu 1.2.3

Command to launche the migration:
virsh migrate --verbose --live --abort-on-error --tunnelled --p2p --auto-converge --copy-storage-inc --xml vm-6160.xml 6160  qemu+tls://<destination_hypervisor>/system

Here is the log output, look at the time elapsed:

root at virt-hv009:~# virsh domjobinfo 6160
Job type:         Unbounded   
Time elapsed:     27518        ms
Data processed:   21.506 GiB
Data remaining:   29.003 GiB
Data total:       50.509 GiB
Memory processed: 0.000 B
Memory remaining: 520.820 MiB
Memory total:     520.820 MiB
File processed:   21.506 GiB
File remaining:   28.494 GiB
File total:       50.000 GiB
Constant pages:   0           
Normal pages:     0           
Normal data:      0.000 B
Expected downtime: 300          ms
Setup time:       6            ms

root at virt-hv009:~# virsh domjobinfo 6160
Job type:         Unbounded   
Time elapsed:     32331        ms
Data processed:   25.475 GiB
Data remaining:   25.034 GiB
Data total:       50.509 GiB
Memory processed: 0.000 B
Memory remaining: 520.820 MiB
Memory total:     520.820 MiB
File processed:   25.475 GiB
File remaining:   24.525 GiB
File total:       50.000 GiB
Constant pages:   0           
Normal pages:     0           
Normal data:      0.000 B
Expected downtime: 300          ms
Setup time:       6            ms

root at virt-hv009:~# virsh domjobinfo 6160
Job type:         Unbounded   
Time elapsed:     49543        ms
Data processed:   50.000 GiB
Data remaining:   520.820 MiB
Data total:       50.509 GiB
Memory processed: 0.000 B
Memory remaining: 520.820 MiB
Memory total:     520.820 MiB
File processed:   50.000 GiB
File remaining:   0.000 B
File total:       50.000 GiB
Constant pages:   0           
Normal pages:     0           
Normal data:      0.000 B
Expected downtime: 300          ms
Setup time:       6            ms


^^^^ Here after 49 sec, the disk has been process, but the memory hasn't started

I skip the logs, but below is the output 5minutes & 56 seconds later, still nothing:

root at virt-hv009:~# virsh domjobinfo 6160
Job type:         Unbounded   
Time elapsed:     356919       ms
Data processed:   50.000 GiB
Data remaining:   520.820 MiB
Data total:       50.509 GiB
Memory processed: 0.000 B
Memory remaining: 520.820 MiB
Memory total:     520.820 MiB
File processed:   50.000 GiB
File remaining:   0.000 B
File total:       50.000 GiB
Constant pages:   0           
Normal pages:     0           
Normal data:      0.000 B
Expected downtime: 300          ms
Setup time:       6            ms

Just after it started to move the memory, but 6 minutes later!

root at virt-hv009:~# virsh domjobinfo 6160
Job type:         Unbounded   
Time elapsed:     360092       ms
Data processed:   50.052 GiB
Data remaining:   453.895 MiB
Data total:       50.509 GiB
Memory processed: 53.224 MiB
Memory remaining: 453.895 MiB
Memory total:     520.820 MiB
Memory bandwidth: 98.172 MiB/s
File processed:   50.000 GiB
File remaining:   0.000 B
File total:       50.000 GiB
Constant pages:   3541        
Normal pages:     13591       
Normal data:      53.090 MiB
Expected downtime: 300          ms
Setup time:       6            ms

root at virt-hv009:~# virsh domjobinfo 6160
error: failed to get domain '6160'
error: Domain not found: no domain with matching name '6160'

Here the migration is done. But what made it wait more than 5 minutes to start moving the memory? Does anyone knows? Or have any idea what could produce this latency.

- Marco
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/libvirt-users/attachments/20160601/fcc2ad4d/attachment.htm>