[Libvir] Start NUMA work
Daniel Veillard
veillard at redhat.com
Tue Sep 11 15:12:17 UTC 2007
Okay enclosed is a first patch to add the new entry point for getting
the available memeory in the NUMA cells:
/**
* virNodeGetCellFreeMemory:
* @conn: pointer to the hypervisor connection
* @freeMems: pointer to the array of unsigned long
* @nbCells: number of entries available in freeMems
*
* This call allows to ask the amount of free memory in each NUMA cell.
* The @freeMems array must be allocated by the caller and will be filled
* with the amounts of free memory in kilobytes for each cell starting
* from cell #0 and up to @nbCells -1 or the number of cell in the Node
* (which can be found using virNodeGetInfo() see the nodes entry in the
* structure).
*
* Returns the number of entries filled in freeMems, or -1 in case of error.
*/
int
virNodeGetCellsFreeMemory(virConnectPtr conn, unsigned long *freeMems,
int nbCells)
based on the feedback, it seems it's better to provide an API checking
a range of cells. This version suggest to always start at cell 0, it could be
extended to start at a base cell number, not a big change, is it needed ?
The patch adds it to the driver interfaces and put the entry point
needed in the xen_internal.c module xenHypervisorNodeGetCellsFreeMemory() .
From there it needs a new function (or set of functions) actually doing
one hypercall to get the free mem for a NUMA cell, and the loop to fill
the array @freeMems. The hard part is of course to set the definitions
and code doing the hypercall:
We will need to check the current hypercall version since this was added
recently, see how xenHypervisorGetSchedulerType() does the versionning,
we will have to write a similar routine , extend xen_op_v2_sys to
add support for the availheap call structures, add the define for
the availheap system call, glue the whole and call the new function
from the loop in xenHypervisorNodeGetCellsFreeMemory() ... this can be
a little fun to debug.
Now for extending virConnectGetCapabilities() it is a bit messy not not
that much. First it's implemented on Xen using xenHypervisorGetCapabilities,
unfortunately it seems the easiest way to get the NUMA capabilities is by
asking though xend. Calling xend_internals.c from xen_internals.c is not
nice, but xenHypervisorGetCapabilities() is actually noty using any
hypervisor call as far as I can see, it's all about opening/parsing
files from /proc and /sys and returning the result as XML, so this could
as well be done in the xend_internals (or xen_unified.c) module.
so we will have a bit of surgery to do, but for the first steps of
writing the patch I would not be too concerned by calling a function
in xend_internal.c from xenHypervisorGetCapabilities (or
xenHypervisorMakeCapabilitiesXML) we will just move those 2 in the end
(only problem may be the access to hv_version variable).
Hope this helps, I will not be online most of the week but I will try
to help when possible :-)
Daniel
--
Red Hat Virtualization group http://redhat.com/virtualization/
Daniel Veillard | virtualization library http://libvirt.org/
veillard at redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/
-------------- next part --------------
Index: include/libvirt/libvirt.h
===================================================================
RCS file: /data/cvs/libxen/include/libvirt/libvirt.h,v
retrieving revision 1.55
diff -u -r1.55 libvirt.h
--- include/libvirt/libvirt.h 10 Sep 2007 09:37:10 -0000 1.55
+++ include/libvirt/libvirt.h 11 Sep 2007 14:43:16 -0000
@@ -582,6 +582,14 @@
int virDomainDetachDevice(virDomainPtr domain, char *xml);
/*
+ * NUMA support
+ */
+
+int virNodeGetCellsFreeMemory(virConnectPtr conn,
+ unsigned long *freeMems,
+ int nbCells);
+
+/*
* Virtual Networks API
*/
Index: include/libvirt/libvirt.h.in
===================================================================
RCS file: /data/cvs/libxen/include/libvirt/libvirt.h.in,v
retrieving revision 1.34
diff -u -r1.34 libvirt.h.in
--- include/libvirt/libvirt.h.in 10 Sep 2007 09:37:10 -0000 1.34
+++ include/libvirt/libvirt.h.in 11 Sep 2007 14:43:16 -0000
@@ -582,6 +582,14 @@
int virDomainDetachDevice(virDomainPtr domain, char *xml);
/*
+ * NUMA support
+ */
+
+int virNodeGetCellsFreeMemory(virConnectPtr conn,
+ unsigned long *freeMems,
+ int nbCells);
+
+/*
* Virtual Networks API
*/
Index: src/driver.h
===================================================================
RCS file: /data/cvs/libxen/src/driver.h,v
retrieving revision 1.35
diff -u -r1.35 driver.h
--- src/driver.h 21 Aug 2007 10:08:12 -0000 1.35
+++ src/driver.h 11 Sep 2007 14:43:16 -0000
@@ -255,6 +255,12 @@
typedef struct _virDriver virDriver;
typedef virDriver *virDriverPtr;
+typedef int
+ (*virDrvNodeGetCellsFreeMemory)
+ (virConnectPtr conn,
+ unsigned long *freeMems,
+ int nbCells);
+
/**
* _virDriver:
*
@@ -322,6 +328,7 @@
virDrvDomainMigrateFinish domainMigrateFinish;
virDrvDomainBlockStats domainBlockStats;
virDrvDomainInterfaceStats domainInterfaceStats;
+ virDrvNodeGetCellsFreeMemory nodeGetCellsFreeMemory;
};
typedef int
Index: src/libvirt.c
===================================================================
RCS file: /data/cvs/libxen/src/libvirt.c,v
retrieving revision 1.98
diff -u -r1.98 libvirt.c
--- src/libvirt.c 10 Sep 2007 09:37:10 -0000 1.98
+++ src/libvirt.c 11 Sep 2007 14:43:16 -0000
@@ -2649,6 +2649,43 @@
}
/**
+ * virNodeGetCellFreeMemory:
+ * @conn: pointer to the hypervisor connection
+ * @freeMems: pointer to the array of unsigned long
+ * @nbCells: number of entries available in freeMems
+ *
+ * This call allows to ask the amount of free memory in each NUMA cell.
+ * The @freeMems array must be allocated by the caller and will be filled
+ * with the amounts of free memory in kilobytes for each cell starting
+ * from cell #0 and up to @nbCells -1 or the number of cell in the Node
+ * (which can be found using virNodeGetInfo() see the nodes entry in the
+ * structure).
+ *
+ * Returns the number of entries filled in freeMems, or -1 in case of error.
+ */
+
+int
+virNodeGetCellsFreeMemory(virConnectPtr conn, unsigned long *freeMems,
+ int nbCells)
+{
+ if (!VIR_IS_CONNECT(conn)) {
+ virLibConnError(conn, VIR_ERR_INVALID_CONN, __FUNCTION__);
+ return (-1);
+ }
+
+ if ((freeMems == NULL) || (nbCells <= 0)) {
+ virLibConnError(conn, VIR_ERR_INVALID_ARG, __FUNCTION__);
+ return (-1);
+ }
+
+ if (conn->driver->nodeGetCellsFreeMemory)
+ return conn->driver->nodeGetCellsFreeMemory (conn, freeMems, nbCells);
+
+ virLibConnError (conn, VIR_ERR_NO_SUPPORT, __FUNCTION__);
+ return -1;
+}
+
+/**
* virNetworkGetConnect:
* @net: pointer to a network
*
Index: src/openvz_driver.c
===================================================================
RCS file: /data/cvs/libxen/src/openvz_driver.c,v
retrieving revision 1.6
diff -u -r1.6 openvz_driver.c
--- src/openvz_driver.c 3 Sep 2007 16:30:00 -0000 1.6
+++ src/openvz_driver.c 11 Sep 2007 14:43:16 -0000
@@ -753,6 +753,7 @@
NULL, /* domainMigrateFinish */
NULL, /* domainBlockStats */
NULL, /* domainInterfaceStats */
+ NULL, /* nodeGetCellsFreeMemory */
};
static virNetworkDriver openvzNetworkDriver = {
Index: src/qemu_driver.c
===================================================================
RCS file: /data/cvs/libxen/src/qemu_driver.c,v
retrieving revision 1.23
diff -u -r1.23 qemu_driver.c
--- src/qemu_driver.c 21 Aug 2007 10:08:12 -0000 1.23
+++ src/qemu_driver.c 11 Sep 2007 14:43:17 -0000
@@ -2667,6 +2667,7 @@
NULL, /* domainMigrateFinish */
NULL, /* domainBlockStats */
NULL, /* domainInterfaceStats */
+ NULL, /* nodeGetCellsFreeMemory */
};
static virNetworkDriver qemuNetworkDriver = {
Index: src/remote_internal.c
===================================================================
RCS file: /data/cvs/libxen/src/remote_internal.c,v
retrieving revision 1.21
diff -u -r1.21 remote_internal.c
--- src/remote_internal.c 21 Aug 2007 10:08:12 -0000 1.21
+++ src/remote_internal.c 11 Sep 2007 14:43:17 -0000
@@ -3117,6 +3117,7 @@
.domainMigrateFinish = remoteDomainMigrateFinish,
.domainBlockStats = remoteDomainBlockStats,
.domainInterfaceStats = remoteDomainInterfaceStats,
+ .nodeGetCellsFreeMemory = NULL,
};
static virNetworkDriver network_driver = {
Index: src/test.c
===================================================================
RCS file: /data/cvs/libxen/src/test.c,v
retrieving revision 1.47
diff -u -r1.47 test.c
--- src/test.c 21 Aug 2007 10:08:12 -0000 1.47
+++ src/test.c 11 Sep 2007 14:43:17 -0000
@@ -1969,6 +1969,7 @@
NULL, /* domainMigrateFinish */
NULL, /* domainBlockStats */
NULL, /* domainInterfaceStats */
+ NULL, /* nodeGetCellsFreeMemory */
};
static virNetworkDriver testNetworkDriver = {
Index: src/xen_internal.c
===================================================================
RCS file: /data/cvs/libxen/src/xen_internal.c,v
retrieving revision 1.94
diff -u -r1.94 xen_internal.c
--- src/xen_internal.c 29 Aug 2007 13:35:15 -0000 1.94
+++ src/xen_internal.c 11 Sep 2007 14:43:17 -0000
@@ -2937,6 +2937,40 @@
}
+/**
+ * xenHypervisorNodeGetCellsFreeMemory:
+ * @conn: pointer to the hypervisor connection
+ * @freeMems: pointer to the array of unsigned long
+ * @nbCells: number of entries available in freeMems
+ *
+ * This call allows to ask the amount of free memory in each NUMA cell.
+ * The @freeMems array must be allocated by the caller and will be filled
+ * with the amounts of free memory in kilobytes for each cell starting
+ * from cell #0 and up to @nbCells -1 or the number of cell in the Node
+ * (which can be found using virNodeGetInfo() see the nodes entry in the
+ * structure).
+ *
+ * Returns the number of entries filled in freeMems, or -1 in case of error.
+ */
+int
+xenHypervisorNodeGetCellsFreeMemory(virConnectPtr conn, unsigned long *freeMems,
+ int nbCells)
+{
+ if ((conn == NULL) || (freeMems == NULL) || (nbCells < 0))
+ return -1;
+
+ /*
+ * TODO:
+ * - get the number of cell in the node
+ * - if not NUMA returns the available memeoy directly in freeMems[0]
+ * return 1
+ * - if NUMA iterates over the cells using a specific hypercall
+ * filling up entries until full or at the end of the NUMA cells
+ */
+ return(-1);
+}
+
+
#ifndef PROXY
/**
* xenHypervisorPauseDomain:
Index: src/xen_internal.h
===================================================================
RCS file: /data/cvs/libxen/src/xen_internal.h,v
retrieving revision 1.23
diff -u -r1.23 xen_internal.h
--- src/xen_internal.h 21 Aug 2007 10:08:12 -0000 1.23
+++ src/xen_internal.h 11 Sep 2007 14:43:17 -0000
@@ -93,6 +93,9 @@
const char *path,
struct _virDomainInterfaceStats *stats);
+int xenHypervisorNodeGetCellsFreeMemory(virConnectPtr conn,
+ unsigned long *freeMems,
+ int nbCells);
#ifdef __cplusplus
}
#endif
Index: src/xen_unified.c
===================================================================
RCS file: /data/cvs/libxen/src/xen_unified.c,v
retrieving revision 1.20
diff -u -r1.20 xen_unified.c
--- src/xen_unified.c 21 Aug 2007 10:08:12 -0000 1.20
+++ src/xen_unified.c 11 Sep 2007 14:43:17 -0000
@@ -1049,6 +1049,19 @@
return -1;
}
+static int
+xenUnifiedNodeGetCellsFreeMemory (virConnectPtr conn, unsigned long *freeMems,
+ int nbCells)
+{
+ GET_PRIVATE (conn);
+
+ if (priv->opened[XEN_UNIFIED_HYPERVISOR_OFFSET])
+ return xenHypervisorNodeGetCellsFreeMemory (conn, freeMems, nbCells);
+
+ xenUnifiedError (conn, VIR_ERR_NO_SUPPORT, __FUNCTION__);
+ return -1;
+}
+
/*----- Register with libvirt.c, and initialise Xen drivers. -----*/
#define VERSION ((DOM0_INTERFACE_VERSION >> 24) * 1000000 + \
@@ -1109,6 +1122,7 @@
.domainMigrateFinish = xenUnifiedDomainMigrateFinish,
.domainBlockStats = xenUnifiedDomainBlockStats,
.domainInterfaceStats = xenUnifiedDomainInterfaceStats,
+ .nodeGetCellsFreeMemory = xenUnifiedNodeGetCellsFreeMemory,
};
/**
More information about the libvir-list
mailing list