[libvirt] qemu: migration: shall we abort migration while the guest is rebooting？

[Zhangbo (Oscar)]

>> Hi all:
>>   Here's the steps we produce the problem:
>>   1 reboot guest with the flag of VIR_DOMAIN_REBOOT_ACPI_POWER_BTN
>>   2 sleep 1 second (so that the guest is still rebooting, although the API
>already returned.)
>>   3 migrate the guest
>>
>>   The problem is that : the guest failed to migrate to the dest, and crashed
>on source side.
>>
>>   We don't bother to dig further into the problem, the root cause we think is
>that we migrate a guest while it's rebooting.
>
>Migration is expected to work no matter what state the guest OS is currently
>in. So the fact that its rebooting should be irrelevant. If anything bad happens
>then its a bug in QEMU most likely
>

Thanks, Daniel! 
This answer helps me finds out the right direction.

There's one exeption that libvirt may also take response for migration/reboot problems. 
Because there's cooperation between libvirt and qemu to complete the reboot job:
1 libvirt send "powerdown" qmp command to qemu, and set vm->priv->fakeReboot to 1.
2 the guest starts to shutdown
3 when the guest got shutoff, qemu sends back "shutdown" monitor message to libvirt
4 libvirt got the message, and checks that vm->priv->fakeReboot is 1, then it sends "reset" qmp command to qemu (otherwise sends "poweroff")

The reboot job is not atomic, libvirt->qemu->libvirt->qemu, they have several times of message sending/receiving.

So, we think of the migration + reboot situation:
1 libvirt send "powerdown" qmp command to qemu, and set vm->priv->fakeReboot to 1.
2 the guest starts to shutdown
3 libvirt migrate the guest to the destination, BUT *priv->fakeReboot is not sended to the dest*
4 the guest is migrated to the dest
5 the guest shutdown completely inside itself, and qemu now sends back "shutdown" monitor message to libvirt
6 libvirt at the dest side got the message, and checks that vm->priv->fakeReboot is 0, then it will send "poweroff" rather than "reset" to qemu, the guest got shutoff rather than rebooted here.

EXPECTED: 
  Guest got rebooted on the dest
EXACT:
  Guest got shutoff on the dest.

So, shall we send privateData to the dest also? I found that libvirt parses privataData when it reloads/starts, we may also send/parse it during migration?

Thanks 

Zhang Bo(Oscar)