[libvirt] [PATCH] qemu: fix migration fail of an auto-placementvm after attached memory to it

[Peter Krempa]

Please configure your e-mail client so that it does not break threads
and also that it does not add spurious newlines between lines of the
reply.

I've trimmed the spurious newlines in the reply.


On Mon, Jul 24, 2017 at 17:11:50 +0800, wang.yi59 at zte.com.cn wrote:
> >On Sat, Jul 22, 2017 at 05:45:59 -0400, Yi Wang wrote:
> >> This patch fix this condition:
> >>   -vm has the "auto" placement in vcpu
> >>   -hot-plug memory with source node "1-3" through attach-device command
> >>   -migrate the vm to a host with only 2 numa node
> >> And the migration will fail with error:
> >> "error: unsupported configuration: NUMA node 2 is unavailable"
> >> >> Signed-off-by: Yi Wang <wang yi59 zte com cn>
> >> Signed-off-by: Xi Xu <xu xi8 zte com cn>
> >> ---
> >>  src/qemu/qemu_process.c | 10 ++++++++++
> >>  1 file changed, 10 insertions(+)
> >> >> diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
> >> index 7b708be..dcc564c 100644
> >> --- a/src/qemu/qemu_process.c
> >> +++ b/src/qemu/qemu_process.c
> >> @@ -5259,6 +5259,16 @@ qemuProcessPrepareDomain(virConnectPtr conn,
> >>              goto cleanup
> >>      }
> >>  
> >> +    VIR_DEBUG("Updating memory source nodes")
> >> +    for (i = 0 i < vm->def->nmems i++) {
> >> +        virDomainMemoryDefPtr mem = vm->def->mems[i]
> >> +        if (priv->autoNodeset && mem && mem->sourceNodes) {
> >> +            virBitmapFree(mem->sourceNodes)
> >> +            if (!(mem->sourceNodes = virBitmapNewCopy(priv->autoNodeset)))
> >> +                goto cleanup
> >
> >This is not correct. This code will be executed even during normal
> >startup and it would remove any manual pinning the user set-up.
> What you said is right, this is indeed a problem that my patch ignored.
> >
> >I think the problem might be that the XML retains the nodesets of
> >automatically placed memory modules while formatting the migratable XML.
> The problem is that the source nodemask specified in the "virsh attach-device"
> XML, which doesn't exist in the destination host. For example:
>     <memory model='dimm'>
>         <target>
>             <size unit='MiB'>1024</size>
>             <node>0</node>
>         </target>
>         <source>
>             <nodemask>1-3</nodemask>
>         </source>
>     </memory>
> The source of the problem is that we must set source nodemask when hot-plug
> memory dimm when the VM has auto placement, or it failed:
> # virsh attach-device centos mem_hp.xml
> error: Failed to attach device from mem_hp.xml
> error: internal error: Advice from numad is needed in case of automatic numa placement

The 'numad' advice is no longer valid if you try to add more memory to
the guest, so you really need to configure things externally with memory
hotplug.

While starting the VM automatic placement for the memory devices works
since we are able to ask numad to accomodate them.

> And after we set source nodemask, it might fail to migrate.
> So, a better way to work around this problem is add a "auto" supported to <source>?

This would mean that the whole VM would need to be re-pinned to the new
nodeset provided by numad, which I don't think is desirable. Adding
'auto' really won't be easy in this case.

You can explore this path though, since it's the only way to do it
automatically.

> Any other suggestion?

Since memory hotplug requires manual setup, you really need to provide
a destination XML on migration, where you can change the nodeset on the
target.

Another option is to use manual pinning since that's the only one that
allows enough flexibility. The automatic placement is really meant only
for basic cases.

Or don't use pinning at all.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Digital signature
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20170724/9a964275/attachment-0001.sig>