[libvirt] [libvirt-python][PATCH] examples: Introduce nodestats example

Martin Kletzander mkletzan at redhat.com
Mon Jul 13 15:02:10 UTC 2015


On Mon, Jun 29, 2015 at 04:53:38PM +0200, Michal Privoznik wrote:
>So, this is an exercise to show libvirt capabilities. Firstly, for
>each host NUMA nodes some statistics are printed out, i.e. total

s/nodes/node/

>memory and free memory. Then, for each running domain, that has memory
>strictly bound to certain host nodes, a small statistics of how much
>memory it takes is printed out too. For instance:
>
>  # ./nodestats.py
>  NUMA stats
>  NUMA nodes:             0       1       2       3
>  MemTotal:               3950    3967    3937    3943
>  MemFree:                434     674     149     216
>  Dom 'gentoo':           1048576 1048576 1048576 1048576
>
>We can see 4 host NUMA nodes, all of them having roughly 4GB of RAM.
>Yeah, some of them has nearly all the memory consumed. Then, there's
>only one running domain, called 'gentoo', and it has 1GB per each NUMA
>node configured.
>
>Signed-off-by: Michal Privoznik <mprivozn at redhat.com>
>---
> examples/nodestats.py | 106 ++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 106 insertions(+)
> create mode 100755 examples/nodestats.py
>
>diff --git a/examples/nodestats.py b/examples/nodestats.py
>new file mode 100755
>index 0000000..dbf5593
>--- /dev/null
>+++ b/examples/nodestats.py
>@@ -0,0 +1,106 @@
>+#!/usr/bin/env python
>+# Print some host NUMA node statistics
>+#
>+# Authors:
>+#   Michal Privoznik <mprivozn at redhat.com

s/$/>/ if you want the authorship here ;)

>+
>+import libvirt
>+import sys
>+from xml.dom import minidom
>+import libxml2
>+
>+class virBitmap:
>+    def __init__(self):
>+        self.bitmap = 0
>+
>+    def setBit(self, offset):
>+        mask = 1 << offset
>+        self.bitmap = self.bitmap | mask
>+
>+    def clearBit(self, offset):
>+        mask = ~(1 << offset)
>+        self.bitmap = self.bitmap & mask
>+
>+    def isSet(self, offset):
>+        mask = 1 << offset
>+        return(self.bitmap & mask)
>+
>+    def setRange(self, start, end):
>+        while (start <= end):
>+            self.setBit(start)
>+            start = start + 1
>+
>+    def parse(self, string):
>+        for s in string.split(','):
>+            list = s.split('-', 2)
>+            start = int(list[0])
>+            if len(list) == 2:
>+                end = int(list[1])
>+            else:
>+                end = start
>+            self.setRange(start, end)
>+

This last function is the only useful one.  You are storing it all in
one integer, so it's not very scalable (well, for python2 it isn't,
python3 stores that in a long that has virtually unlimited size).

Anyway, you are only storing nodeset here, for everything else you are
using arrays.  And the nodeset won't be very big.  I'd rather you
changed this to a normal array.

>+def xpath_eval(ctxt, path):
>+    res = ctxt.xpathEval(path)
>+    if res is None or len(res) == 0:
>+        value = None
>+    else:
>+        value = res[0].content
>+    return value
>+
>+try:
>+    conn = libvirt.openReadOnly(None)
>+except libvirt.libvirtError:
>+    print('Failed to connect to the hypervisor')
>+    sys.exit(1)
>+
>+try:
>+    capsXML = conn.getCapabilities()
>+except libvirt.libvirtError:
>+    print('Failed to request capabilities')
>+    sys.exit(1)
>+
>+caps = minidom.parseString(capsXML)
>+cells = caps.getElementsByTagName('cells')[0]
>+
>+nodesIDs = [ int(proc.getAttribute('id'))
>+             for proc in cells.getElementsByTagName('cell') ]
>+
>+nodesMem = [ conn.getMemoryStats(int(proc))
>+             for proc in nodesIDs]
>+

You could do { proc.getAttribute('id') :
conn.getMemoryStats(int(proc.getAttribute('id))) } and have it in a
dictionary, so it's easier to list through, but I'd prefer more
readable:

nodes = {}
for p in cells.getElementsByTagName('cell'):
    i = p.getAttribute('id')
    nodes[i] = conn.getMemoryStats(i);

>+doms = conn.listAllDomains(libvirt.VIR_CONNECT_LIST_DOMAINS_ACTIVE)
>+domsStrict = [ proc
>+               for proc in doms
>+               if proc.numaParameters()['numa_mode'] == libvirt.VIR_DOMAIN_NUMATUNE_MEM_STRICT ]
>+

This will list most of the domains even though there will be no info
for them.  You need to check whether there is any memnode with strict
memory assigned.  Although skipping everything else still feels kinda
fishy.  Maybe there should be another field for unspecified memnodes.

Also if you run this with a domain with:

  <memnode cellid='0' mode='strict' nodeset='0-3'/>

You report the allocated memory 4 times.

>+domsStrictCfg = {}
>+
>+for dom in domsStrict:
>+    xmlStr = dom.XMLDesc()
>+    doc = libxml2.parseDoc(xmlStr)
>+    ctxt = doc.xpathNewContext()
>+
>+    domsStrictCfg[dom] = [ 0  for node in nodesIDs ]
>+
>+    for memnode in ctxt.xpathEval("/domain/numatune/memnode"):
>+        ctxt.setContextNode(memnode)
>+        cellid = xpath_eval(ctxt, "@cellid")
>+        mode = xpath_eval(ctxt, "@mode")
>+        nodeset = xpath_eval(ctxt, "@nodeset")
>+
>+        bitmap = virBitmap()
>+        bitmap.parse(nodeset)
>+        for node in nodesIDs:
>+            if bitmap.isSet(int(node)):
>+                mem = xpath_eval(ctxt, "/domain/cpu/numa/cell[@id='%s']/@memory" % cellid)
>+                domsStrictCfg[dom][int(node)] += int(mem)
>+
>+print("NUMA stats")
>+print("NUMA nodes:\t\t" + "\t".join(str(node) for node in nodesIDs))
>+print("MemTotal:\t\t" + "\t".join(str(i.get('total') / 1024) for i in nodesMem))
>+print("MemFree:\t\t" + "\t".join(str(i.get('free') / 1024) for i in nodesMem))
>+

Use one TAB so it's intuitive to parse without empty fields (be it
with 'cut' or importing to 'libreoffice', doesn't matter).

Also this produces ugly output on python3, you might want to round the
number or divide it using '//' instead of '/'.

>+for dom in domsStrictCfg:
>+    sys.stdout.write("Dom '%s':\t\t" % dom.name())
>+    print("\t".join(map(str, domsStrictCfg[dom][:])))

No need to copy the list, plus generators are preferred to mapping.
And why do you do sys.stdout.write() once, but print() after that?
You can do 'print "asdf", ' in python2 or 'print("asdf", end="")' in
python3 if you don't want newline at the end, but even better not use
it at all.

for k, v in domsStrictCfg.items():
    print("Dom '%s':\t%s" % (k.name(), "\t".join([str(x) for x in v])))
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20150713/8e12dfec/attachment-0001.sig>


More information about the libvir-list mailing list