[Libvir] Re: [PATCH 1/2] virDomainMigrate implementation (Xen only, no remote, no qemu, no virsh)
Anthony Liguori
anthony at codemonkey.ws
Mon Jul 16 15:18:10 UTC 2007
Richard W.M. Jones wrote:
> To cut to the chase, this is the proposed additional libvirt call to
> support migration. Please also read my explanation below.
>
> /**
> * virDomainMigrate:
> * @domain: a domain object
> * @dconn: destination host (a connection object)
> * @flags: flags
> * @dname: (optional) rename domain to this at destination
> * @hostname: (optional) remote hostname as seen from the source host
> * @params: linked list of hypervisor-specific parameters
> *
> * Migrate the domain object from its current host to the destination
> * host given by dconn (a connection to the destination host).
> *
> * Flags may be one of more of the following:
> * VIR_MIGRATE_LIVE Attempt a live migration.
> *
> * If a hypervisor supports renaming domains during migration,
> * then you may set the dname parameter to the new name (otherwise
> * it keeps the same name). If this is not supported by the
> * hypervisor, dname must be NULL or else you will get an error.
> *
> * Since typically the two hypervisors connect directly to each
> * other in order to perform the migration, you may need to specify
> * a hostname, which is the hostname or IP address of the destination
> * host as seen from the source host. If in doubt, leave this as
> * NULL and libvirt will attempt to work out the correct hostname.
> *
> * Params is a linked list of hypervisor-specific parameters. Each
> * element is a virMigrateParamPtr containing the following fields:
> * name Parameter name being set.
> * value A union expressing the value.
> * value.strv A string value.
> * value.longv A long value.
> * next Next in linked list (or NULL for end of list).
> *
> * Parameter names for Xen are:
> * VIR_MIGRATE_XEN_PORT (long) Override the default port number.
> * VIR_MIGRATE_XEN_RESOURCE (long) Set maximum resource usage (Mbps).
> *
> * Set params to NULL if you do not want to pass any hypervisor-specific
> * parameters.
> *
> * Returns the new domain object if the migration was successful,
> * or NULL in case of error.
> */
>
> As discussed previously on this list, you need to have libvirt
> connections to both the source and destination hosts.
>
> I've tried to separate out what I believe will be common features of
> migration across hypervisors, and what is currently supported by Xen.
>
> What I think will be common features are:
> * live migration
> * direct host<->host connections
> * renaming domains during migration
Absolutely not! The issue only exists if you allow the guests to exist
on both systems during the migration. In KVM, this shouldn't be the
case. The domain on the target should only ever be visible after the
domain has successfully completed.
Localhost migration works fine today in KVM without domain renaming.
Regards,
Anthony Liguori
> These are supported with explicit parameters. Drivers should check the
> parameters and any which are not supported should be rejected (eg. Xen
> cannot rename a domain when it is migrating, although this seems like it
> ought to be a common thing to want to do -- to prevent name clashes on
> the destination host which would otherwise make it impossible to migrate
> a domain).
>
> The explicit parameters include a general "flags" parameter, which we
> can extend with other boolean flags later. For host<->host connections
> you'll want some way to specify the hostname / IP address of the
> destination host as seen at the source. In the remote management case
> it's not always so easy to work this out. We can try using
> virConnectGetHostname, but we also allow the caller to override.
>
> On the other hand, there will be some hypervisor-specific features, and
> these are enabled through a linked list of parameters. For Xen these
> include setting port and resource usage. I guess other hypervisors will
> have their own parameters -- eg. security settings.
>
> In the current (Xen) implementation, any parameters which it doesn't
> understand are rejected with VIR_ERR_NO_SUPPORT.
>
> Rich.
>
>
> ------------------------------------------------------------------------
>
> Index: include/libvirt/libvirt.h
> ===================================================================
> RCS file: /data/cvs/libvirt/include/libvirt/libvirt.h,v
> retrieving revision 1.49
> diff -u -p -r1.49 libvirt.h
> --- include/libvirt/libvirt.h 9 Jul 2007 12:41:30 -0000 1.49
> +++ include/libvirt/libvirt.h 16 Jul 2007 12:31:44 -0000
> @@ -209,6 +209,30 @@ int virDomainSetSchedulerParameters (vir
> virSchedParameterPtr params,
> int nparams);
>
> +/* Hypervisor-specific migration parameters. */
> +enum virMigrateParamName {
> + VIR_MIGRATE_XEN_PORT = 1,
> + VIR_MIGRATE_XEN_RESOURCE = 2
> +};
> +
> +struct virMigrateParam {
> + struct virMigrateParam *next;
> + enum virMigrateParamName name;
> + union {
> + const char *strv;
> + long longv;
> + } value;
> +};
> +typedef struct virMigrateParam *virMigrateParamPtr;
> +
> + /* Domain migration flags. */
> +#define VIR_MIGRATE_LIVE 1
> +
> +/* Domain migration. */
> +virDomainPtr virDomainMigrate (virDomainPtr domain, virConnectPtr dconn,
> + unsigned long flags, const char *dname,
> + const char *hostname, virMigrateParamPtr params);
> +
> /**
> * VIR_NODEINFO_MAXCPUS:
> * @nodeinfo: virNodeInfo instance
> Index: include/libvirt/libvirt.h.in
> ===================================================================
> RCS file: /data/cvs/libvirt/include/libvirt/libvirt.h.in,v
> retrieving revision 1.30
> diff -u -p -r1.30 libvirt.h.in
> --- include/libvirt/libvirt.h.in 26 Jun 2007 11:42:46 -0000 1.30
> +++ include/libvirt/libvirt.h.in 16 Jul 2007 12:31:45 -0000
> @@ -209,6 +209,30 @@ int virDomainSetSchedulerParameters (vir
> virSchedParameterPtr params,
> int nparams);
>
> +/* Hypervisor-specific migration parameters. */
> +enum virMigrateParamName {
> + VIR_MIGRATE_XEN_PORT = 1,
> + VIR_MIGRATE_XEN_RESOURCE = 2
> +};
> +
> +struct virMigrateParam {
> + struct virMigrateParam *next;
> + enum virMigrateParamName name;
> + union {
> + const char *strv;
> + long longv;
> + } value;
> +};
> +typedef struct virMigrateParam *virMigrateParamPtr;
> +
> + /* Domain migration flags. */
> +#define VIR_MIGRATE_LIVE 1
> +
> +/* Domain migration. */
> +virDomainPtr virDomainMigrate (virDomainPtr domain, virConnectPtr dconn,
> + unsigned long flags, const char *dname,
> + const char *hostname, virMigrateParamPtr params);
> +
> /**
> * VIR_NODEINFO_MAXCPUS:
> * @nodeinfo: virNodeInfo instance
> Index: src/driver.h
> ===================================================================
> RCS file: /data/cvs/libvirt/src/driver.h,v
> retrieving revision 1.30
> diff -u -p -r1.30 driver.h
> --- src/driver.h 26 Jun 2007 22:56:14 -0000 1.30
> +++ src/driver.h 16 Jul 2007 12:31:45 -0000
> @@ -180,6 +180,15 @@ typedef int
> virSchedParameterPtr params,
> int nparams);
>
> +typedef virDomainPtr
> + (*virDrvDomainMigrate)
> + (virDomainPtr domain,
> + virConnectPtr dconn,
> + unsigned long flags,
> + const char *dname,
> + const char *hostname,
> + virMigrateParamPtr params);
> +
> typedef struct _virDriver virDriver;
> typedef virDriver *virDriverPtr;
>
> @@ -244,6 +253,7 @@ struct _virDriver {
> virDrvDomainGetSchedulerType domainGetSchedulerType;
> virDrvDomainGetSchedulerParameters domainGetSchedulerParameters;
> virDrvDomainSetSchedulerParameters domainSetSchedulerParameters;
> + virDrvDomainMigrate domainMigrate;
> };
>
> typedef int
> Index: src/libvirt.c
> ===================================================================
> RCS file: /data/cvs/libvirt/src/libvirt.c,v
> retrieving revision 1.88
> diff -u -p -r1.88 libvirt.c
> --- src/libvirt.c 12 Jul 2007 08:34:51 -0000 1.88
> +++ src/libvirt.c 16 Jul 2007 12:31:47 -0000
> @@ -1662,6 +1662,96 @@ virDomainGetXMLDesc(virDomainPtr domain,
> }
>
> /**
> + * virDomainMigrate:
> + * @domain: a domain object
> + * @dconn: destination host (a connection object)
> + * @flags: flags
> + * @dname: (optional) rename domain to this at destination
> + * @hostname: (optional) remote hostname as seen from the source host
> + * @params: linked list of hypervisor-specific parameters
> + *
> + * Migrate the domain object from its current host to the destination
> + * host given by dconn (a connection to the destination host).
> + *
> + * Flags may be one of more of the following:
> + * VIR_MIGRATE_LIVE Attempt a live migration.
> + *
> + * If a hypervisor supports renaming domains during migration,
> + * then you may set the dname parameter to the new name (otherwise
> + * it keeps the same name). If this is not supported by the
> + * hypervisor, dname must be NULL or else you will get an error.
> + *
> + * Since typically the two hypervisors connect directly to each
> + * other in order to perform the migration, you may need to specify
> + * a hostname, which is the hostname or IP address of the destination
> + * host as seen from the source host. If in doubt, leave this as
> + * NULL and libvirt will attempt to work out the correct hostname.
> + *
> + * Params is a linked list of hypervisor-specific parameters. Each
> + * element is a virMigrateParamPtr containing the following fields:
> + * name Parameter name being set.
> + * value A union expressing the value.
> + * value.strv A string value.
> + * value.longv A long value.
> + * next Next in linked list (or NULL for end of list).
> + *
> + * Parameter names for Xen are:
> + * VIR_MIGRATE_XEN_PORT (long) Override the default port number.
> + * VIR_MIGRATE_XEN_RESOURCE (long) Set maximum resource usage (Mbps).
> + *
> + * Set params to NULL if you do not want to pass any hypervisor-specific
> + * parameters.
> + *
> + * Returns the new domain object if the migration was successful,
> + * or NULL in case of error.
> + */
> +virDomainPtr
> +virDomainMigrate (virDomainPtr domain,
> + virConnectPtr dconn,
> + unsigned long flags,
> + const char *dname,
> + const char *hostname,
> + virMigrateParamPtr params)
> +{
> + virConnectPtr conn;
> + virDomainPtr ddomain;
> + char *nchostname = NULL;
> + DEBUG("domain=%p, dconn=%p, flags=%lu, dname=%s, hostname=%s, params=%p",
> + domain, dconn, flags, dname, hostname, params);
> +
> + if (!VIR_IS_DOMAIN (domain)) {
> + virLibDomainError(domain, VIR_ERR_INVALID_DOMAIN, __FUNCTION__);
> + return NULL;
> + }
> + conn = domain->conn; /* Source connection. */
> + if (!VIR_IS_CONNECT (dconn)) {
> + virLibConnError (conn, VIR_ERR_INVALID_CONN, __FUNCTION__);
> + return NULL;
> + }
> +
> + /* Check that migration is supported. */
> + if (!conn->driver->domainMigrate) {
> + virLibConnError (conn, VIR_ERR_NO_SUPPORT, __FUNCTION__);
> + return NULL;
> + }
> +
> + /* Synthesize a hostname if one is not given. */
> + if (!hostname) {
> + nchostname = virConnectGetHostname (dconn);
> + if (!nchostname) return NULL;
> + }
> +
> + /* Try to migrate. */
> + ddomain = conn->driver->domainMigrate (domain, dconn, flags,
> + dname,
> + hostname ? hostname : nchostname,
> + params);
> +
> + if (nchostname) free (nchostname);
> + return ddomain;
> +}
> +
> +/**
> * virNodeGetInfo:
> * @conn: pointer to the hypervisor connection
> * @info: pointer to a virNodeInfo structure allocated by the user
> Index: src/libvirt_sym.version
> ===================================================================
> RCS file: /data/cvs/libvirt/src/libvirt_sym.version,v
> retrieving revision 1.25
> diff -u -p -r1.25 libvirt_sym.version
> --- src/libvirt_sym.version 26 Jun 2007 22:56:14 -0000 1.25
> +++ src/libvirt_sym.version 16 Jul 2007 12:31:47 -0000
> @@ -69,6 +69,8 @@
> virDomainAttachDevice;
> virDomainDetachDevice;
>
> + virDomainMigrate;
> +
> virNetworkGetConnect;
> virConnectNumOfNetworks;
> virConnectListNetworks;
> Index: src/qemu_driver.c
> ===================================================================
> RCS file: /data/cvs/libvirt/src/qemu_driver.c,v
> retrieving revision 1.8
> diff -u -p -r1.8 qemu_driver.c
> --- src/qemu_driver.c 12 Jul 2007 15:09:01 -0000 1.8
> +++ src/qemu_driver.c 16 Jul 2007 12:31:49 -0000
> @@ -2507,6 +2507,7 @@ static virDriver qemuDriver = {
> NULL, /* domainGetSchedulerType */
> NULL, /* domainGetSchedulerParameters */
> NULL, /* domainSetSchedulerParameters */
> + NULL, /* domainMigrate */
> };
>
> static virNetworkDriver qemuNetworkDriver = {
> Index: src/test.c
> ===================================================================
> RCS file: /data/cvs/libvirt/src/test.c,v
> retrieving revision 1.41
> diff -u -p -r1.41 test.c
> --- src/test.c 6 Jul 2007 15:02:09 -0000 1.41
> +++ src/test.c 16 Jul 2007 12:31:50 -0000
> @@ -144,6 +144,7 @@ static virDriver testDriver = {
> NULL, /* domainGetSchedulerType */
> NULL, /* domainGetSchedulerParameters */
> NULL, /* domainSetSchedulerParameters */
> + NULL, /* domainMigrate */
> };
>
> /* Per-connection private data. */
> Index: src/xen_unified.c
> ===================================================================
> RCS file: /data/cvs/libvirt/src/xen_unified.c,v
> retrieving revision 1.17
> diff -u -p -r1.17 xen_unified.c
> --- src/xen_unified.c 12 Jul 2007 08:34:51 -0000 1.17
> +++ src/xen_unified.c 16 Jul 2007 12:31:50 -0000
> @@ -791,6 +791,24 @@ xenUnifiedDomainDumpXML (virDomainPtr do
> return NULL;
> }
>
> +static virDomainPtr
> +xenUnifiedDomainMigrate (virDomainPtr dom,
> + virConnectPtr dconn,
> + unsigned long flags,
> + const char *dname,
> + const char *hostname,
> + virMigrateParamPtr params)
> +{
> + GET_PRIVATE(dom->conn);
> +
> + if (priv->opened[XEN_UNIFIED_XEND_OFFSET])
> + return xenDaemonDomainMigrate (dom, dconn, flags,
> + dname, hostname, params);
> +
> + xenUnifiedError (dom->conn, VIR_ERR_NO_SUPPORT, __FUNCTION__);
> + return NULL;
> +}
> +
> static int
> xenUnifiedListDefinedDomains (virConnectPtr conn, char **const names,
> int maxnames)
> @@ -1002,6 +1020,7 @@ static virDriver xenUnifiedDriver = {
> .domainGetSchedulerType = xenUnifiedDomainGetSchedulerType,
> .domainGetSchedulerParameters = xenUnifiedDomainGetSchedulerParameters,
> .domainSetSchedulerParameters = xenUnifiedDomainSetSchedulerParameters,
> + .domainMigrate = xenUnifiedDomainMigrate,
> };
>
> /**
> Index: src/xend_internal.c
> ===================================================================
> RCS file: /data/cvs/libvirt/src/xend_internal.c,v
> retrieving revision 1.129
> diff -u -p -r1.129 xend_internal.c
> --- src/xend_internal.c 9 Jul 2007 11:24:52 -0000 1.129
> +++ src/xend_internal.c 16 Jul 2007 12:31:52 -0000
> @@ -3126,6 +3126,77 @@ xenDaemonDetachDevice(virDomainPtr domai
> }
>
>
> +virDomainPtr
> +xenDaemonDomainMigrate (virDomainPtr domain,
> + virConnectPtr dconn,
> + unsigned long flags,
> + const char *dname,
> + const char *hostname,
> + virMigrateParamPtr params)
> +{
> + /* Upper layers have already checked domain, dconn, etc. */
> + virConnectPtr conn = domain->conn;
> + /* NB: Passing port=0, resource=0 to xend means it ignores
> + * those parameters. However this is somewhat specific to
> + * the internals of the xend Python code. (XXX).
> + */
> + char port[16] = "0";
> + char resource[16] = "0";
> + char live[16] = "0";
> + int ret;
> +
> + /* Xen doesn't support renaming domains during migration. */
> + if (dname) {
> + virXendError (conn, VIR_ERR_NO_SUPPORT,
> + "xenDaemonDomainMigrate: Xen does not support renaming domains during migration");
> + return NULL;
> + }
> +
> + /* Check the parameters and set variables as necessary. */
> + for (; params; params = params->next) {
> + switch (params->name) {
> + case VIR_MIGRATE_XEN_PORT:
> + snprintf (port, sizeof port, "%ld", params->value.longv);
> + break;
> + case VIR_MIGRATE_XEN_RESOURCE:
> + snprintf (resource, sizeof resource, "%ld", params->value.longv);
> + break;
> + default:
> + virXendError (conn, VIR_ERR_NO_SUPPORT,
> + "xenDaemonDomainMigrate: unsupported parameter");
> + return NULL;
> + }
> + }
> +
> + /* Check the flags. */
> + if ((flags & VIR_MIGRATE_LIVE)) {
> + strcpy (live, "1");
> + flags &= ~VIR_MIGRATE_LIVE;
> + }
> + if (flags != 0) {
> + virXendError (conn, VIR_ERR_NO_SUPPORT,
> + "xenDaemonDomainMigrate: unsupported flag");
> + return NULL;
> + }
> +
> + /* Make the call. */
> + ret = xend_op (domain->conn, domain->name,
> + "op", "migrate",
> + "destination", hostname,
> + "live", live,
> + "resource", resource,
> + "port", port,
> + NULL);
> + if (ret == -1)
> + return NULL;
> +
> + printf ("migration op returned\n");
> +
> + /* Look for the new domain on the destination host. */
> + return xenDaemonLookupByName (dconn, domain->name);
> +}
> +
> +
> virDomainPtr xenDaemonDomainDefineXML(virConnectPtr conn, const char *xmlDesc) {
> int ret;
> char *sexpr;
> Index: src/xend_internal.h
> ===================================================================
> RCS file: /data/cvs/libvirt/src/xend_internal.h,v
> retrieving revision 1.31
> diff -u -p -r1.31 xend_internal.h
> --- src/xend_internal.h 6 Jul 2007 15:11:22 -0000 1.31
> +++ src/xend_internal.h 16 Jul 2007 12:31:53 -0000
> @@ -219,6 +219,7 @@ int xenDaemonInit (void);
> virDomainPtr xenDaemonLookupByID(virConnectPtr conn, int id);
> virDomainPtr xenDaemonLookupByUUID(virConnectPtr conn, const unsigned char *uuid);
> virDomainPtr xenDaemonLookupByName(virConnectPtr conn, const char *domname);
> +virDomainPtr xenDaemonDomainMigrate (virDomainPtr domain, virConnectPtr dconn, unsigned long flags, const char *dname, const char *hostname, virMigrateParamPtr params);
>
> #ifdef __cplusplus
> }
>
More information about the libvir-list
mailing list