[libvirt] [PATCH v6 1/4] numa: describe siblings distances within cells

Fri Nov 10 17:29:59 UTC 2017

On 11/02/2017 09:47 AM, Wim Ten Have wrote:
> From: Wim ten Have <wim.ten.have at oracle.com>
> 
> Add support for describing NUMA distances in a domain's <numa> <cell>
> XML description.
> 
> Below is an example of a 4 node setup:
> 
> <cpu>
> <numa>
> <cell id='0' cpus='0-3' memory='2097152' unit='KiB'>
> <distances>
> <sibling id='0' value='10'/>
> <sibling id='1' value='21'/>
> <sibling id='2' value='31'/>
> <sibling id='3' value='21'/>
> </distances>
> </cell>
> <cell id='1' cpus='4-7' memory='2097152' unit='KiB'>
> <distances>
> <sibling id='0' value='21'/>
> <sibling id='1' value='10'/>
> <sibling id='2' value='21'/>
> <sibling id='3' value='31'/>
> </distances>
> </cell>
> <cell id='2' cpus='8-11' memory='2097152' unit='KiB'>
> <distances>
> <sibling id='0' value='31'/>
> <sibling id='1' value='21'/>
> <sibling id='2' value='10'/>
> <sibling id='3' value='21'/>
> </distances>
> <cell id='3' cpus='12-15' memory='2097152' unit='KiB'>
> <distances>
> <sibling id='0' value='21'/>
> <sibling id='1' value='31'/>
> <sibling id='2' value='21'/>
> <sibling id='3' value='10'/>
> </distances>
> </cell>
> </numa>
> </cpu>
> 
> A <cell> defines a NUMA node. <distances> describes the NUMA distance
> from the <cell> to the other NUMA nodes (the <sibling>s). For example,
> in above XML description, the distance between NUMA node0 <cell id='0'
> ...> and NUMA node2 <sibling id='2' ...> is 31.
> 
> Valid distance value are '10 <= value <= 255'. A distance value of 10
> represents the distance to the node itself. A distance value of 20
> represents the default value for remote nodes but other values are
> possible depending on the physical topology of the system.
> 
> When distances are not fully described, any missing sibling distance
> values will default to 10 for local nodes and 20 for remote nodes.
> 
> If distance is given for A -> B, then we default B -> A to the same
> value instead of 20.
> 
> Signed-off-by: Wim ten Have <wim.ten.have at oracle.com>
> Reviewed-by: Daniel P. Berrange <berrange at redhat.com>
> ---
> Changes on v1:
> - Add changes to docs/formatdomain.html.in describing schema update.
> Changes on v2:
> - Automatically apply distance symmetry maintaining cell <-> sibling.
> - Check for maximum '255' on numaDistanceValue.
> - Automatically complete empty distance ranges.
> - Check that sibling_id's are in range with cell identifiers.
> - Allow non-contiguous ranges, starting from any node id.
> - Respect parameters as ATTRIBUTE_NONNULL fix functions and callers.
> - Add and apply topology for LOCAL_DISTANCE=10 and REMOTE_DISTANCE=20.
> Changes on v3:
> - Add UNREACHABLE if one locality is unreachable from another.
> - Add code cleanup aligning function naming in a separated patch.
> - Add numa related driver code in a separated patch.
> - Remove <choice> from numaDistanceValue schema/basictypes.rng
> - Correct doc changes.
> Changes on v4:
> - Fix symmetry error under virDomainNumaDefNodeDistanceParseXML()
> Changes on v5:
> - Apply doc suggestions.
> - Apply virReportError() message suggestions.
> - Remove redundant sanity checks from virDomainNumaDefNodeDistanceParseXML().
> ---
> docs/formatdomain.html.in | 63 ++++++++++++++-
> docs/schemas/basictypes.rng | 7 ++
> docs/schemas/cputypes.rng | 18 +++++
> src/conf/numa_conf.c | 191 +++++++++++++++++++++++++++++++++++++++++++-
> 4 files changed, 275 insertions(+), 4 deletions(-)
> 
> diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
> index 4f28dce35..df2f17f5d 100644
> --- a/docs/formatdomain.html.in
> +++ b/docs/formatdomain.html.in
> @@ -1529,7 +1529,68 @@
> 
> 
> 
> - This guest NUMA specification is currently available only for QEMU/KVM.
> + This guest NUMA specification is currently available only for
> + QEMU/KVM and Xen. Whereas Xen driver also allows for a distinct
> + description of NUMA arranged <code>sibling</code> <code>cell</code>
> + <code>distances</code> Since 3.9.0.
> + 
> +
> + 
> + Under NUMA hardware architecture, distinct resources such as memory
> + create a designated distance between <code>cell</code> and
> + <code>siblings</code> that now can be described with the help of
> + <code>distances</code>. A detailed description can be found within
> + the ACPI (Advanced Configuration and Power Interface Specification)
> + within the chapter explaining the system's SLIT (System Locality
> + Distance Information Table).
> + 

Sorry for not reading this a bit closer when reviewing V5. The patch looks good 
otherwise and has already been ACKed, but what do you think about the below diff 
that rewords these changes? If you think it is ok, I'll squash it before pushing.

Regards,
Jim

diff --git a/docs/formatdomain.html.in b/docs/formatdomain.html.in
index 7ca0d88b6..47c43d066 100644
--- a/docs/formatdomain.html.in
+++ b/docs/formatdomain.html.in
@@ -1530,19 +1530,19 @@

      <p>
        This guest NUMA specification is currently available only for
-      QEMU/KVM and Xen.  Whereas Xen driver also allows for a distinct
-      description of NUMA arranged <code>sibling</code> <code>cell</code>
-      <code>distances</code> <span class="since">Since 3.9.0</span>.
+      QEMU/KVM and Xen.
      </p>

      <p>
-      Under NUMA hardware architecture, distinct resources such as memory
-      create a designated distance between <code>cell</code> and
-      <code>siblings</code> that now can be described with the help of
-      <code>distances</code>. A detailed description can be found within
-      the ACPI (Advanced Configuration and Power Interface Specification)
-      within the chapter explaining the system's SLIT (System Locality
-      Distance Information Table).
+      A NUMA hardware architecture supports the notion of distances
+      between NUMA cells. <span class="since">Since 3.10.0</span> it
+      is possible to define the distance between NUMA cells using the
+      <code>distances</code> element within a NUMA <code>cell</code>
+      description. The <code>sibling</code> sub-element is used to
+      specify the distance value between sibling NUMA cells. For more
+      details, see the chapter explaining the system's SLIT (System
+      Locality Information Table) within the ACPI (Advanced
+      Configuration and Power Interface) specification.
      </p>

  <pre>
@@ -1588,7 +1588,8 @@
  ...</pre>

      <p>
-      Under Xen driver, if no <code>distances</code> are given to describe
+      Describing distances between NUMA cells is currently only supported
+      by Xen. If no <code>distances</code> are given to describe
        the SLIT data between different cells, it will default to a scheme
        using 10 for local and 20 for remote distances.
      </p>


