[libvirt] [RFC][scale] new API for querying domains stats

[Li Wei]

Hi Richard,

Thanks for your comment!

On 08/04/2014 04:39 PM, Richard W.M. Jones wrote:
> On Mon, Aug 04, 2014 at 11:38:41AM +0800, Li Wei wrote:
>> Hi,
>>
>> On 07/22/2014 03:25 PM, Richard W.M. Jones wrote:
>>>
>>> Did anything come of this discussion, and/or is someone working on this?
>>
>> I am working on an API to query block stats in a bulk style and proposed an
>> API as follow:
>>
>> virDomainBlockStatsBulkFlags(virDomainPtr dom,
>> 			     virTypedParameterPtr params,
>> 			     int nparams,
>> 			     int ndisks,
>> 			     unsigned int flags)
>>
>> @dom: pointer to domain object
>> @params: an array of typed param to be populated with block stats
>> @nparams: how many params used for each block device
>> @ndisks: how many block devices to query
>> @flags: flags to filter block devices (not used for now)
>>
>> Returns -1 in case of error, 0 in case of success.
>> with params == NULL, nparams == -1, ndisks == 1, return number of params for each block device.
>> with params == NULL, nparams == -1, ndisks == -1, return number of disks in the domain.
>>
>> A typical usage of this API should be:
>> nparams = virDomainBlockStatsBulkFlags(dom, NULL, -1, 1, 0);
>> ndisks = virDomainBlockStatsBulkFlags(dom, NULL, -1, -1, 0);
>>
>> params = VIR_ALLOC_N(params, nparams * ndisks);
>>
>> ret = virDomainBlockStatsBulkFlags(dom, params, nparams, ndisks, 0);
>>
>> ... do something with params
>>
>> VIR_FREE(params);
>>
>> With this bulk API, virt-top can updates in a short interval for a domain with a lot of disks.
>> Any comments?
> 
> I think this works OK for the case where you have 1 domains with
> lots of disks.
> 
> However if you have a large number of domains each with 1 or 2
> disks I think you would have the same problem as currently.

Yes, it is.

> 
> Is it possible to design an API that can work across all domains
> in a single call?

How about the following API:

int virConnectGetAllBlockStats(virConnectPtr conn,
				virDomainPtr domain,
				virDomainBlockBulkStatsPtr *stats,
				unsigned int flags);
@conn: pointer to libvirt connection
@domain: pointer to the domain to be queried, NULL for all domains
@stats: array of virDomainBlockBulkStats struct(see below) to be populated
@flags: filter flags
Return the number of virDomainBlockBulkStats populated.

where virDomainBlockBulkStats defined as:

struct _virDomainBlockBulkStats {
    virDomainPtr domain;	 /* domain the block stats belongs to */
    virTypedParameterPtr params; /* params to store block stats */
    unsigned int nparams;	 /* how many params used for each block stats */
    unsigned int ndisks;	 /* how many block stats in this domain */
};

Note:
1. because the API allocate memory to store stats, the caller need to free it after use.
2. to distinguish each block stats in a domain, we need use a param to store block device name.

> 
>> PS:
>> It seems we need a bunch of bulk APIs to query stats, I wonder if I can submit a patchset for each
>> bulk API or must supply all the bulk APIs in one patchset?
> 
> Whichever is easiest to review.  I suspect that smaller patches, each
> containing a single new API, will be simpler to review, but that's
> just my opinion.

I prefer this way also.

Thanks,
Li Wei

> 
> Rich.
>