[libvirt] [PATCH RFC 0/5] support cpu bandwidth in libvirt

[Wen Congyang]

At 06/16/2011 03:47 AM, Adam Litke Write:
> 
> 
> On 06/15/2011 01:24 AM, Wen Congyang wrote:
>> I need this feature immediately after CFS bandwidth patchset is merged into
>> upstream kernel. So I prepare this patchset, and post it here for reviewing
>> before CFS bandwidth is merged into upstream kernel.
>>
>> quota is an optional attribute specified in microseconds not a percentage of period.
>>
>> TODO:
>> 1. quota should be in the range [1000, 18446744073709551(2^64/1000)] or less than 0.
>>    But I only limit it less or equal than 2^64/1000. Does anyone have a better
>>    way to limit quota?
> 
> What are the semantics of quota < 0, of quota == 0?  How are you storing

>From Documentation/scheduler/sched-bwc.txt:
=======================================
A group with cpu.cfs_quota_us as -1 indicates that the group has infinite
bandwidth, which means that it is not bandwidth controlled.

Writing any negative value to cpu.cfs_quota_us will turn the group into
an infinite bandwidth group. Reading cpu.cfs_quota_us for an infinite
bandwidth group will always return -1.
=======================================

quota == 0 is invalid.

> such a large signed integer in the kernel?  I will guess that any value

In the kernel, the value of quota is stored in uint64_t:
=======================================
struct cfs_bandwidth {
#ifdef CONFIG_CFS_BANDWIDTH
	raw_spinlock_t lock;
	ktime_t period;
	u64 quota;             <========== here is u64, not s64
	u64 runtime;
	u64 runtime_expires;
	s64 hierarchal_quota;

	int idle;
	struct hrtimer period_timer, slack_timer;
	struct list_head throttled_cfs_rq;

	/* statistics */
	int nr_periods, nr_throttled;
	u64 throttled_time;

#endif
};
=======================================

The unit of quota in kernel is ns, but the value we write to cpu.cfs_quota_us is us.
So the max value we can write is 2^64/1000.

In the kernel, if quota is ~0ULL, it means unlimited.

> less than 0 means: no limit, and 0 means: no quota (ie. the cgroup
> cannot run on the cpu).

If the kernel does not support CFS bandwidth, quota is 0 and we do not allow user to
modify it. 0 is internal value, and the user does not know it.

> 
> I am not sure what the libvirt convention for dealing with a situation
> like this is.  I think you have two options: 1) override an invalid
> value (ie. 1) to represent the case (n < 0) or, 2) use a struct to
> represent the quota:

I do not think so. The value that user can specify should be the similar the user
can write it to cpu.cfs_quota_us. If so, the user can use it easily.

> 
> struct _virCfsQuota {
>     unsigned long long val;
>     int unlimited;
> };
> 
> When the quota is unlimited, .unlimited == 1 and val is undefined.
> Otherwise, the quota is in val.
> 
> 
>>
>> Wen Congyang (5):
>>   cgroup: Implement cpu.cfs_period_us and cpu.cfs_quota_us tuning API
>>   Update XML Schema for new entries
>>   qemu: Implement period and quota tunable XML configuration and
>>     parsing.
>>   qemu: Implement cfs_period and cfs_quota's modification
>>   doc: Add documentation for new cputune elements period and quota
>>
>>  docs/formatdomain.html.in                       |   19 +++
>>  docs/schemas/domain.rng                         |   25 ++++-
>>  src/conf/domain_conf.c                          |   20 +++-
>>  src/conf/domain_conf.h                          |    2 +
>>  src/libvirt_private.syms                        |    4 +
>>  src/qemu/qemu_cgroup.c                          |   43 +++++-
>>  src/qemu/qemu_driver.c                          |  162 +++++++++++++++++++----
>>  src/util/cgroup.c                               |   81 +++++++++++-
>>  src/util/cgroup.h                               |    6 +
>>  tests/qemuxml2argvdata/qemuxml2argv-cputune.xml |    2 +
>>  10 files changed, 328 insertions(+), 36 deletions(-)
>