[Ovirt-devel] Re: [Patch] Experimental Graphs Backend Code
Mohammed Morsi
mmorsi at redhat.com
Thu May 29 16:17:22 UTC 2008
mark wagner wrote:
> Comments inline
>
> Hugh O. Brock wrote:
>>> # retrieves data used by history graphs
>>> def history_graph
>>> - today = DateTime.now
>>> + target = params[:target]
>>> + today = Time.now
>>> + requestList = [ StatsRequest.new(@pool.id, target, 0, "used", today.to_i - 3600, 3600, 0),
>>> + StatsRequest.new(@pool.id, target, 0, "peak", today.to_i - 3600, 3600, 0) ]
>>> dates = [ Date::ABBR_MONTHNAMES[today.month] + ' ' + today.day.to_s ]
>>> 1.upto(6){ |x| # TODO get # of days from wui
>>> dte = today - x
>>> dates.push ( Date::ABBR_MONTHNAMES[dte.month] + ' ' + dte.day.to_s )
>>> + requestList.push ( StatsRequest.new (@pool.id, target, 0, "used", dte.to_i - 3600, 3600, 0),
>>> + StatsRequest.new (@pool.id, target, 0, "peak", dte.to_i - 3600, 3600, 0) )
>>>
> Not sure what exactly you are trying to get in the above chunk of
> code. Doesn't look like its something gathered by collectd at this
> point in time. If you let me know what data it is, we can see about
> adding it to the collectd configuration.
So this is invoked via a callback when each history graph is being
rendered. The history graphs are the line graphs on the 2nd row of the
hardware pool summary page, each describing the 7 day history (at some
point the time frame will be able to be set via the WUI) for the
particular target the user selects to view. Here I am passing the
following data into your StatsRequest API.
* node - id of the pool the user is looking at
* devClass - 'targets', get passed in from the WUI. oOne of the
following string values: cpu, io, system (last is for 'overall load').
* instance - 0 (what should i set this to?)
* counter - "used" and "peak" for the two data sets respectively
* starttime / duration - i'm just looking for one day's worth of
data. I loop through the past seven days, trying to collect the
statistic for one particular day one each pass. It seems that I'm
getting alot more data than I need back
* precision - 0 as you mentioned in your email
Are all (or any) of these right?
>>> }
>>> dates.reverse! # want in ascending order
>>> -
>>> - target = params[:target]
>>> - peakvalues = nil
>>> - avgvalues = nil
>>> - if target == 'host_usage'
>>> - peakvalues = [95.97, 91.80, 88.16, 86.64, 99.14, 75.14, 85.69] # TODO real values!
>>> - avgvalues = [3.39, 2.83, 1.61, 0.00, 4.56, 1.23, 5.32] # TODO real values!
>>> - elsif target == 'storage_usage'
>>> - peakvalues = [11.12, 22.29, 99.12, 13.23, 54.32, 17.91, 50.1] # TODO real values!
>>> - avgvalues = [19.23, 19.23, 19.23, 29.12, 68.96, 43.11, 0.1] # TODO real values!
>>> - elsif target == 'vm_pool_usage_history'
>>> - peakvalues = [42, 42, 42, 42, 42, 42, 42] # TODO real values!
>>> - avgvalues = [0, 0, 0, 0, 0, 0, 0] # TODO real values!
>>> - elsif target == 'overall_load'
>>> - peakvalues = [19.68, 20.08, 19.84, 17.76, 0.0, 14.78, 9.71] # TODO real values!
>>> - avgvalues = [0, 1, 2, 4, 8, 16, 32] # TODO real values!
>>> - end
>>> + requestList.reverse!
>>> +
>>> + statsList = getStatsData?( requestList )
>>> + statsList.each { |stat|
>>> + devClass = stat.get_devClass?
>>> + counter = stat.get_counter?
>>> + value = stat.get_value?.to_i + 20
>>> + if devClass == target
>>> + if counter == "used"
>>> + @avg_history[:values].push value
>>> + else
>>> + #elsif counter == "peak"
>>> + @peak_history[:values].push value
>>> + end
>>> + end
>>> + }
>>>
>>>
> The interface is changing for retrieving data. I've created a
> StatsDataList object that will contain a lot of the "duplicate" info
> thats in the StatsData objects. This will allow the StatsData objects
> to be smaller. In addition, I will be passing back a list of lists.
> The API you currently have basically just passes back a big array of
> data objects that need to get parsed. With the list of
> StatsDataLists, you are guaranteed that each StatsDataList only
> contains the data for a unique counter (so this list only contains
> data for node3/cpu-0/cpu-idle).
>
So when we have the new API, I will loop through the list in a manner
like so?
list1 = getStatsData?( requestList )
list1.each { |sublist|
sublist.each{ |item|
...
}
}
What specifically will be contained in each list. Eg. is list1 a list of
unique devices containing a sublist of all unique counter values for
that device? Also once again, I'm a fan of hashes in ruby, they allow
you to do almost everything an array does, but it is alot easier for the
client to lookup information. Perhaps we can create hashes indexed by
device identifier / counter name or whatever else is appropriate.
>
>>> graph_object = {
>>> :timepoints => dates,
>>> @@ -171,14 +186,14 @@ class HardwareController < ApplicationController
>>> [
>>> {
>>> :name => target + "peak",
>>> - :values => peakvalues,
>>> - :stroke => @peak_color,
>>> + :values => @peak_history[:values],
>>> + :stroke => @peak_history[:color],
>>> :strokeWidth => 1
>>> },
>>> {
>>> :name => target + "average",
>>> - :values => avgvalues,
>>> - :stroke => @average_color,
>>> + :values => @avg_history[:values],
>>> + :stroke => @avg_history[:color],
>>> :strokeWidth => 1
>>> }
>>> ]
>>>
>
> We typically use rrd to provide the "average" values, we can also get
> min and max values. I'll look into adding calls for those
How will these be referenced when I call them via your API. eg. is
"average" equivalent to the "used" counter? What are the counters for
"max" and "min"
>>> @@ -186,21 +201,25 @@ class HardwareController < ApplicationController
>>> render :json => graph_object
>>> end
>>>
>>> - def network_traffic_graph
>>> + def snapshot_graph
>>> target = params[:target]
>>> - network_load = nil
>>> - if target == 'in'
>>> - network_load = @network_traffic['in']
>>> + snapshot = nil
>>> + if target == 'overall_load'
>>> + snapshot = @snapshots[:overall_load]
>>> + elsif target == 'cpu'
>>> + snapshot = @snapshots[:cpu]
>>> + elsif target == 'in'
>>> + snapshot = @snapshots[:in]
>>> elsif target == 'out'
>>> - network_load = @network_traffic['out']
>>> + snapshot = @snapshots[:out]
>>> elsif target == 'io'
>>> - network_load = @network_traffic['io']
>>> + snapshot = @snapshots[:io]
>>> end
>>>
> I need to modify the collectd configs to include network and the others .
>
>> -
>>
>> This looks exactly like what we talked about -- understand you still
>> need feedback from Mark regarding the right way to pull this info out.
>>
>>
>>
>>> - network_load_remaining = 1024 - network_load
>>> + snapshot_remaining = 1024 - snapshot
>>>
>>> color = 'blue'
>>> - color = 'red' if (network_load.to_f / 1024.to_f) > 0.75 # 3/4 is the critical boundry for now
>>> + color = 'red' if (snapshot.to_f / 1024.to_f) > 0.75 # 3/4 is the critical boundry for now
>>>
>>> graph_object = {
>>> :timepoints => [],
>>> @@ -208,14 +227,14 @@ class HardwareController < ApplicationController
>>> [
>>> {
>>> :name => target,
>>> - :values => [network_load],
>>> + :values => [snapshot],
>>> :fill => color,
>>> :stroke => 'lightgray',
>>> :strokeWidth => 1
>>> },
>>> {
>>> :name => target + "remaining",
>>> - :values => [network_load_remaining],
>>> + :values => [snapshot_remaining],
>>> :fill => 'white',
>>> :stroke => 'lightgray',
>>> :strokeWidth => 1
>>> @@ -395,23 +414,56 @@ class HardwareController < ApplicationController
>>> @perm_obj = @pool
>>> @current_pool_id=@pool.id
>>>
>>> - # TODO pull real values in
>>> - @available_memory = 18
>>> - @used_memory = 62
>>> -
>>> - @available_storage = 183
>>> - @used_storage = 61
>>> -
>>> - @available_vms = 1
>>> - @used_vms = 26
>>> -
>>> - @peak_color = 'red'
>>> - @average_color = 'blue'
>>> + # availability graphs - used
>>> + @used = {:cpu => 0, :memory => 0, :vms => 0}
>>> + @pool.sub_vm_resource_pools.each { |svrp| @used[:cpu] += svrp.allocated_resources[:current][:cpus] }
>>> + @pool.sub_vm_resource_pools.each { |svrp| @used[:memory] += svrp.allocated_resources[:current][:memory] }
>>> + @pool.sub_vm_resource_pools.each { |svrp| @used[:vms] += svrp.allocated_resources[:current][:vms] }
>>> +
>>> + # availability graphs - total
>>> + @total = {:cpu => 0, :memory => 0, :vms => 0}
>>> + @total[:cpu] = @pool.total_resources[:cpus]
>>> + @total[:memory] = @pool.total_resources[:memory]
>>> + @total[:vms] = @pool.total_resources[:vms]
>>> + @total.each_key { |k| @total[k] = 0 if @total[k] == nil }
>>> +
>>> + # availability graphs - available
>>> + @available = {}
>>> + @available[:cpu] = (@total[:cpu] - @used[:cpu]).abs
>>> + @available[:memory] = (@total[:memory] - @used[:memory]).abs
>>> + @available[:vms] = 5 # TODO ?
>>> +
>>> + # history graphs
>>> + @peak_history = { :color => 'red', :values => [] }
>>> + @avg_history = { :color => 'blue', :values => [] }
>>> +
>>> + # snapshot graphs
>>> + ret_time = Time.now.to_i - 3600
>>> + @snapshots = { :overall_load => 0, :cpu => 0, :in => 0, :out => 0, :io => 0 }
>>> + requestList = []
>>> + requestList << StatsRequest.new(@pool.id, "system", 0, "used", ret_time, 3600, 0)
>>> + requestList << StatsRequest.new(@pool.id, "cpu", 0, "used", ret_time, 3600, 0)
>>> + requestList << StatsRequest.new(@pool.id, "in", 0, "used", ret_time, 3600, 0)
>>> + requestList << StatsRequest.new(@pool.id, "out", 0, "used", ret_time, 3600, 0)
>>> + requestList << StatsRequest.new(@pool.id, "io", 0, "used", ret_time, 3600, 0)
>>>
> these calls don't look right I am assuming that @pool.id returns the
> machine name. however "io", "in", "out" do not appear to be devClasses.
Previous questions apply here. When invoking your API should I just pass
in a set (configurable?) machine name? Should I use pool id for the
instance id? "in" and "out" are the network device classes (i can change
the names if needed) and "io" is for the i/o statistics / graphs.
>
> -mark
-Mo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/ovirt-devel/attachments/20080529/0b15e779/attachment.htm>
More information about the ovirt-devel
mailing list