[libvirt] CPU topology 'sockets' handling guest vs host

Daniel P. Berrange berrange at redhat.com
Mon Mar 26 15:11:08 UTC 2012


On Mon, Mar 26, 2012 at 05:08:05PM +0200, Jiri Denemark wrote:
> On Mon, Mar 26, 2012 at 15:42:58 +0100, Daniel P. Berrange wrote:
> > On my x86_64 host I have a pair of Quad core CPUs, each in a separate
> > NUMA node. The virsh capabilities
> > topology data reports this:
> > 
> >   # virsh capabilities | xmllint  --xpath /capabilities/host/cpu -
> >   <cpu>
> >       <arch>x86_64</arch>
> >       <model>Opteron_G3</model>
> >       <vendor>AMD</vendor>
> >       <topology sockets="1" cores="4" threads="1"/>
> >       <feature name="osvw"/>
> >       <feature name="3dnowprefetch"/>
> >       <feature name="cr8legacy"/>
> >       <feature name="extapic"/>
> >       <feature name="cmp_legacy"/>
> >       <feature name="3dnow"/>
> >       <feature name="3dnowext"/>
> >       <feature name="pdpe1gb"/>
> >       <feature name="fxsr_opt"/>
> >       <feature name="mmxext"/>
> >       <feature name="ht"/>
> >       <feature name="vme"/>
> >     </cpu>
> >   # virsh capabilities | xmllint  --xpath /capabilities/host/topology -
> >   <topology>
> >       <cells num="2">
> >         <cell id="0">
> >           <cpus num="4">
> >             <cpu id="0"/>
> >             <cpu id="1"/>
> >             <cpu id="2"/>
> >             <cpu id="3"/>
> >           </cpus>
> >         </cell>
> >         <cell id="1">
> >           <cpus num="4">
> >             <cpu id="4"/>
> >             <cpu id="5"/>
> >             <cpu id="6"/>
> >             <cpu id="7"/>
> >           </cpus>
> >         </cell>
> >       </cells>
> >   </topology>
> > 
> > Note, it is reporting sockets=1, because sockets is the number of sockets
> > *per* NUMA node.
> > 
> > 
> > Now I try to figure the guest to match the host using:
> > 
> >   <cpu>
> >     <topology sockets='1' cores='4' threads='1'/>
> >     <numa>
> >       <cell cpus='0-3' memory='512000'/>
> >       <cell cpus='4-7' memory='512000'/>
> >     </numa>
> >   </cpu>
> > 
> > And I get:
> > 
> >   error: Maximum CPUs greater than topology limit
> > 
> > So, the XML checker is mistaking 'sockets' as the total number of sockets,
> > rather than the per-node socket count. We need to fix this bogus check
> 
> I guess what we actually want to do is to report total number of sockets in
> host cpu xml. Sockets per NUMA node has been proven to be a bad decision and
> we should not let it infect other areas.

No, we can't change that - we explicitly fixed that a while back
because it breaks the VIR_NODEINFO_MAXCPUS macro to do that.


commit ac9dd4a676f21b5e3ca6dbe0526f2a6709072beb
Author: Jiri Denemark <jdenemar at redhat.com>
Date:   Wed Nov 24 11:25:19 2010 +0100

    Fix host CPU counting on unusual NUMA topologies
    
    The nodeinfo structure includes
    
        nodes   : the number of NUMA cell, 1 for uniform mem access
        sockets : number of CPU socket per node
        cores   : number of core per socket
        threads : number of threads per core
    
    which does not work well for NUMA topologies where each node does not
    consist of integral number of CPU sockets.
    
    We also have VIR_NODEINFO_MAXCPUS macro in public libvirt.h which
    computes maximum number of CPUs as (nodes * sockets * cores * threads).
    
    As a result, we can't just change sockets to report total number of
    sockets instead of sockets per node. This would probably be the easiest
    since I doubt anyone is using the field directly. But because of the
    macro, some apps might be using sockets indirectly.
    
    This patch leaves sockets to be the number of CPU sockets per node (and
    fixes qemu driver to comply with this) on machines where sockets can be
    divided by nodes. If we can't divide sockets by nodes, we behave as if
    there was just one NUMA node containing all sockets. Apps interested in
    NUMA should consult capabilities XML, which is what they probably do
    anyway.
    
    This way, the only case in which apps that care about NUMA may break is
    on machines with funky NUMA topology. And there is a chance libvirt
    wasn't able to start any guests on those machines anyway (although it
    depends on the topology, total number of CPUs and kernel version).
    Nothing changes at all for apps that don't care about NUMA.


Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|




More information about the libvir-list mailing list