[libvirt PATCH 5/6] Make PATHs unique for a VM object instance

Shaju Abraham shaju.abraham at nutanix.com
Fri Feb 7 04:04:43 UTC 2020


> This is the really critical problem with localhost migration.

   > Appending the domain-id looks "simple" but this is a significant
    >behavioural / functional change for applications, and I don't think
    >it can fully solve the problem either.

    >This is changing thue paths used in various places where libvirt
    >internally generates unique paths (eg QMP socket, huge page or
    >file based memory paths, and defaults used for auto-filling
    >device paths (<channel> if not specified).

    >Some of these paths are functionally important to external
    >applications and cannot be changed in this way. eg stuff
    >integrating with QEMU can be expecting certain memory backing
    >file paths, or certain <channel> paths & is liable to break
    >if we change the naming convention.

    >For sake of argument, lets assume we can changing the naming
    >convention without breaking anything...

Agree on the need to keep the path names unmodified. I will try to design
a new approach using symlinks.  

    > If localhost migration is only usable in a small subset
    >scenarios, I'm not convinced it is worth the support
    >burden. Rarely used & tested features are liable to
    >bitrot, and migration is already a serious point of
    >instability in QEMU in general.

The local migration is a much needed feature in data centers. At present, an upgrade
of the hypervisor in a data center requires a large maintenance window , due to the
migration of the VMs. QEMU  already have features like "x-ignore-shared" which is
almost like zero copy migration . Since Libvirt lacks support for local migration, we are not
able to take advantage of  these features. 

Regards
Shaju

On 2/5/20, 11:03 PM, "Daniel P. Berrangé" <berrange at redhat.com> wrote:

    On Mon, Feb 03, 2020 at 12:43:32PM +0000, Daniel P. Berrangé wrote:
    > From: Shaju Abraham <shaju.abraham at nutanix.com>
    > 
    > There are various config paths that a VM uses. The monitor paths and
    > other lib paths are examples. These paths are tied to the VM name or
    > UUID. The local migration breaks the assumption that there will be only
    > one VM with a unique UUID and name. During local migrations there can be
    > multiple VMs with same name and UUID in the same host. Append the
    > domain-id field to the path so that there is no duplication of path
    > names.
    
    This is the really critical problem with localhost migration.
    
    Appending the domain-id looks "simple" but this is a significant
    behavioural / functional change for applications, and I don't think
    it can fully solve the problem either.
    
    This is changing thue paths used in various places where libvirt
    internally generates unique paths (eg QMP socket, huge page or
    file based memory paths, and defaults used for auto-filling
    device paths (<channel> if not specified).
    
    Some of these paths are functionally important to external
    applications and cannot be changed in this way. eg stuff
    integrating with QEMU can be expecting certain memory backing
    file paths, or certain <channel> paths & is liable to break
    if we change the naming convention.
    
    For sake of argument, lets assume we can changing the naming
    convention without breaking anything...
    
    This only applies to paths libvirt generates at VM startup.
    
    There are plenty of configuration elements in the guest XML
    that are end user / mgmt app defined, and reference paths in
    the host OS.
    
    For example <graphics>, <serial>, <console>, <channel>,
    all support UNIX domain sockets and TCP sockets. A UNIX
    domain socket cannot be listened on by multiple VMs
    at once. If the UNIX socket is in client mode, we cannot
    assume the thing QEMU is connecting to allows multiple
    concurrent connections. eg 2 QEMU's could have their
    <serial> connected together over a UNIX socket pair.
    Similarly if automatic TCP port assignment is not used
    we cannot have multiple QEMU's listening on the same
    host.
    
    One answer is to say that localhost migration is just
    not supported for such VMs, but I don't find that very
    convincing because the UNIX domain socket configs
    affected are in common use.
    
    If localhost migration is only usable in a small subset
    scenarios, I'm not convinced it is worth the support
    burden. Rarely used & tested features are liable to
    bitrot, and migration is already a serious point of
    instability in QEMU in general.
    
    > Signed-off-by: Shaju Abraham <shaju.abraham at nutanix.com>
    > ---
    >  src/qemu/qemu_conf.c   |  4 ++--
    >  src/qemu/qemu_domain.c | 16 ++++++++++------
    >  2 files changed, 12 insertions(+), 8 deletions(-)
    > 
    > diff --git a/src/qemu/qemu_conf.c b/src/qemu/qemu_conf.c
    > index 00801ef01b..6769736d58 100644
    > --- a/src/qemu/qemu_conf.c
    > +++ b/src/qemu/qemu_conf.c
    > @@ -1894,7 +1894,7 @@ qemuGetDomainHugepagePath(const virDomainDef *def,
    >      char *ret = NULL;
    >  
    >      if (base && domPath)
    > -        ret = g_strdup_printf("%s/%s", base, domPath);
    > +        ret = g_strdup_printf("%s/%s-%d", base, domPath, def->id);
    >      return ret;
    >  }
    >  
    > @@ -1962,7 +1962,7 @@ qemuGetMemoryBackingDomainPath(const virDomainDef *def,
    >          return -1;
    >  
    >      qemuGetMemoryBackingBasePath(cfg, &base);
    > -    *path = g_strdup_printf("%s/%s", base, shortName);
    > +    *path = g_strdup_printf("%s/%s-%d", base, shortName, def->id);
    >  
    >      return 0;
    >  }
    > diff --git a/src/qemu/qemu_domain.c b/src/qemu/qemu_domain.c
    > index b0c0e1a19b..002c092cf8 100644
    > --- a/src/qemu/qemu_domain.c
    > +++ b/src/qemu/qemu_domain.c
    > @@ -2127,11 +2127,13 @@ qemuDomainSetPrivatePathsOld(virQEMUDriverPtr driver,
    >      virQEMUDriverConfigPtr cfg = virQEMUDriverGetConfig(driver);
    >  
    >      if (!priv->libDir)
    > -        priv->libDir = g_strdup_printf("%s/domain-%s", cfg->libDir, vm->def->name);
    > +        priv->libDir = g_strdup_printf("%s/domain-%s-%d", cfg->libDir,
    > +                                       vm->def->name, vm->def->id);
    >  
    >      if (!priv->channelTargetDir)
    > -        priv->channelTargetDir = g_strdup_printf("%s/domain-%s",
    > -                                                 cfg->channelTargetDir, vm->def->name);
    > +        priv->channelTargetDir = g_strdup_printf("%s/domain-%s-%d",
    > +                                                 cfg->channelTargetDir,
    > +                                                 vm->def->name, vm->def->id);
    >  
    >      virObjectUnref(cfg);
    >  }
    > @@ -2150,11 +2152,13 @@ qemuDomainSetPrivatePaths(virQEMUDriverPtr driver,
    >          goto cleanup;
    >  
    >      if (!priv->libDir)
    > -        priv->libDir = g_strdup_printf("%s/domain-%s", cfg->libDir, domname);
    > +        priv->libDir = g_strdup_printf("%s/domain-%s-%d", cfg->libDir, domname,
    > +                                       vm->def->id);
    >  
    >      if (!priv->channelTargetDir)
    > -        priv->channelTargetDir = g_strdup_printf("%s/domain-%s",
    > -                                                 cfg->channelTargetDir, domname);
    > +        priv->channelTargetDir = g_strdup_printf("%s/domain-%s-%d",
    > +                                                 cfg->channelTargetDir,
    > +                                                 domname, vm->def->id);
    >  
    >      ret = 0;
    >   cleanup:
    > -- 
    > 2.24.1
    > 
    
    Regards,
    Daniel
    -- 
    |: https://urldefense.proofpoint.com/v2/url?u=https-3A__berrange.com&d=DwIFaQ&c=s883GpUCOChKOHiocYtGcg&r=sY-XeNqcuy_ruBQ9T7A2LmG6ktyYXXSxRB1ljkxMepI&m=wOg64DtCNMIOAHZJH27aA6STzZD7wKkkU0Oxf6pQ4UQ&s=q_RMo9hPZMHyB8Gn1lmP1kIcS8tLqtSysmxdlRONxLc&e=       -o-    https://urldefense.proofpoint.com/v2/url?u=https-3A__www.flickr.com_photos_dberrange&d=DwIFaQ&c=s883GpUCOChKOHiocYtGcg&r=sY-XeNqcuy_ruBQ9T7A2LmG6ktyYXXSxRB1ljkxMepI&m=wOg64DtCNMIOAHZJH27aA6STzZD7wKkkU0Oxf6pQ4UQ&s=tz40Lg7IggJKQG15YYagc7KtfvNxph37V_0LSgCobEo&e=  :|
    |: https://urldefense.proofpoint.com/v2/url?u=https-3A__libvirt.org&d=DwIFaQ&c=s883GpUCOChKOHiocYtGcg&r=sY-XeNqcuy_ruBQ9T7A2LmG6ktyYXXSxRB1ljkxMepI&m=wOg64DtCNMIOAHZJH27aA6STzZD7wKkkU0Oxf6pQ4UQ&s=eQgHUoWy1TZFOChYnRiZITxMTYr1ORZuSUXJGUa3cDQ&e=          -o-            https://urldefense.proofpoint.com/v2/url?u=https-3A__fstop138.berrange.com&d=DwIFaQ&c=s883GpUCOChKOHiocYtGcg&r=sY-XeNqcuy_ruBQ9T7A2LmG6ktyYXXSxRB1ljkxMepI&m=wOg64DtCNMIOAHZJH27aA6STzZD7wKkkU0Oxf6pQ4UQ&s=Z07AZnscj4rQkk9oK1T0jIrkz7UYlDgauYiWoV5KIBg&e=  :|
    |: https://urldefense.proofpoint.com/v2/url?u=https-3A__entangle-2Dphoto.org&d=DwIFaQ&c=s883GpUCOChKOHiocYtGcg&r=sY-XeNqcuy_ruBQ9T7A2LmG6ktyYXXSxRB1ljkxMepI&m=wOg64DtCNMIOAHZJH27aA6STzZD7wKkkU0Oxf6pQ4UQ&s=Nushxar5SvxzJ8cnPTxTn3NouuEV-oh-IiH8Pl98kvo&e=     -o-    https://urldefense.proofpoint.com/v2/url?u=https-3A__www.instagram.com_dberrange&d=DwIFaQ&c=s883GpUCOChKOHiocYtGcg&r=sY-XeNqcuy_ruBQ9T7A2LmG6ktyYXXSxRB1ljkxMepI&m=wOg64DtCNMIOAHZJH27aA6STzZD7wKkkU0Oxf6pQ4UQ&s=qVatQRJpoD_LqBVLXJBMHEqo3MKpEIXseNm4maPEt0Q&e=  :|
    
    





More information about the libvir-list mailing list