[libvirt] [PATCH 3/3] virCaps: Expose distance between host NUMA nodes

[Michal Privoznik]

On 02.06.2014 17:55, Daniel P. Berrange wrote:
> On Mon, Jun 02, 2014 at 02:16:00PM +0200, Michal Privoznik wrote:
>> If user or management application wants to create a guest,
>> it may be useful to know the cost of internode latencies
>> before the guest resources are pinned. For example:
>>
>> <capabilities>
>>
>>    <host>
>>      ...
>>      <topology>
>>        <cells num='2'>
>>          <cell id='0'>
>>            <memory unit='KiB'>4004132</memory>
>>            <cell id='0' distance='10'/>
>>            <cell id='1' distance='20'/>
>
> I'd be a little more comfortable if we didn't use a <cell>
> within a <cell>. Perhaps lets use 'sibling' as the name
> instead and group the elements. eg could we do
>
>    <distances>
>       <sibling id="0" value="10"/>
>       <sibling id="1" value="20'/>
>    </distance>
>
>>      </topology>
>>      ...
>>    </host>
>>    ...
>> </capabilities>
>>
>> we can see the distance from node1 to node0 is 20 and within nodes 10.
>
> One thing with having the data under each <cell> is that we're
> actually reporting twice as much as we need to. ie the distance
> between cell N and M is reported under both N and M. A different
> option would be todo reporting at the toplevel within <topology>
> eg
>
>   <distances>
>      <siblings distance="10">
>         <cell id="0"/>
>         <cell id="1"/>
>      </siblings>
>   </distance>
>
> I'm not sure whether doing this is worth while or not though ?

I'm not sure this is the correct approach. The XML snippet suggest that 
from node0 to node1 the distance is only 10 (which can't be correct, 
since it's not the same node). I know you made the numbers up, but 
still. It's unclear from the XML what the groups are. Consider this NUMA 
topology:

    0   1   2
0  10  20  20
1  20  10  20
2  20  20  10

which is pretty common topology if you have 3 NUMA nodes. Now, what 
would the <distance/> XML look like?

<distance>
   <siblings distance="10">
     <cell id="0"/>
   </siblings>
   <siblings distance="10">
     <cell id="1"/>
   </siblings>
   <siblings distance="10">
     <cell id="2"/>
   </siblings>
   <siblings distance="20">
     <cell id="0"/>
     <cell id="1"/>
     <cell id="2"/>
   </siblings>
</distance>

The XML reflects that distance within a node itself the is 10, while 
it's 20 in case of any two nodes different to each other. But it's not 
visible at first glance why there are multiple <cell/> elements with the 
same @id. Moreover, we can't join the first three <siblings/> into a 
single element because it would have a different meaning then that the 
last one.

So I'll go with your first suggestion.

Michal