[libvirt] RFC: add Intel CMT feature support
Ren, Qiaowei
qiaowei.ren at intel.com
Mon Jun 29 14:18:14 UTC 2015
> -----Original Message-----
> From: Peter Krempa [mailto:pkrempa at redhat.com]
> Sent: Monday, June 29, 2015 5:44 PM
> To: Ren, Qiaowei
> Cc: libvir-list at redhat.com
> Subject: Re: [libvirt] RFC: add Intel CMT feature support
>
> On Sat, Jun 27, 2015 at 03:45:21 +0000, Ren, Qiaowei wrote:
> > Hi All,
> >
> > Some Intel processor families (e.g. the Intel Xeon processor E5 v3 family)
> introduced some PQos (Platform Qos) features to monitor or control shared
> resource.
> >
> > - CMT (Cache Monitoring Technology): measure the usage of cache by
> applications running on the platform.
> > - CAT (Cache Allocation Technology): enable an OS or Hypervisor/VMM to
> specify the amount of cache space into which an application can fill.
> > - MBM (Memory Bandwidth Monitoring): build on the CMT infrastructure to
> allow monitoring of bandwidth from one level of the cache hierarchy to the next
> - in this case focusing on the L3 cache, which is typically backed directly by
> system memory. As a result of this implementation, memory bandwidth can be
> monitored.
> >
> > For more information, see "Intel 64 and IA-32 Architectures Software
> Developer's Manual".
> >
> > Among these PQos features, currently CMT patches has been merged into
> Linux kernel mainline. So this patch series proposed in this mail will focus on
> CMT feature support in libvirt.
> >
> > At the library API layer, I plan on adding cache related field into virDomainInfo
> and virNodeInfo:
> >
> > Add "cache" member into virNodeInfo to get total size of cache in one node.
> > struct virNodeInfo {
> > ...
> > unsigned int cores; /* number of cores per socket, total number of
> > processors in case of unusual NUMA topology*/
> > unsigned int threads; /* number of threads per core, 1 in case of
> > unusual numa topology */
> > + unsigned int cache; /* cache size in bytes */
>
> We don't allow changing/adding members to public structures. You probably will
> be better of by adding this info to the capabilities XML.
>
Ok. So I have to modify capabilities XML and struct virCaps to add total cache size related field.
> > };
> >
> > Add "cacheOcc" member into virDomainInfo to get cache occupancy for one
> VM.
> > struct virDomainInfo {
> > ...
> > unsigned short nrVirtCpu; /* the number of virtual CPUs for the domain */
> > unsigned long long cpuTime; /* the CPU time used in nanoseconds */
> > + unsigned long long cacheOcc; /* the cache occupancy in Bytes */
>
> Neither this structure can be modified.
>
> > };
> >
> > With this change, those applications like OpenStack based on libvirt can collect
> cache usage for metering and monitoring (e.g. any VM using more than X% of
> cache).
>
> You can use the bulk stats API (virDomainListGetStats) to add this field since that
> uses the typed parameter approach which can be extended.
>
Got it. I will try to extend virDomainListGetStats API.
> >
> > Because CMT implementation in Linux kernel is based on perf mechanism and
> there is no nice and public API library for perf, I have to wrapper system call for
> perf in libvirt to use CMT feature, and enable/disable perf event for CMT when
> VM is created/destroyed.
> >
> > Any thoughts on this plan before I start submitting code patches? Do you think
> whether I need add several new APIs and XML element for CMT feature (or more
> PQos features)?
> >
Thanks,
Qiaowei
More information about the libvir-list
mailing list