<div dir="ltr"><div><div><div>Hi kostya.<br><br></div>I'm still interested in this topic. Let's discuss implementations details on the github issue, But AFAIR, we agreed with safchain that we could rely on third party go lib that polls counter interfaces : <br><br><a href="https://github.com/shirou/gopsutil/tree/master/net">https://github.com/shirou/gopsutil/tree/master/net</a><br><br></div>And add interesting data to node's metadata.<br><br></div>Mat<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Nov 9, 2016 at 5:18 PM, Kostya <span dir="ltr"><<a href="mailto:konstantin.dorfman@gmail.com" target="_blank">konstantin.dorfman@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hello Mathieu, <div>Do you want to discuss design/implementation ideas on <a href="https://github.com/skydive-project/skydive/issues/109" target="_blank">https://github.com/skydive-<wbr>project/skydive/issues/109</a> ? This will be very helpful for me. </div><div><br></div><div>We need some general API to get/calculate bandwidth of the edge, it could be implemented as sum of every flow's ABbytes for specific node (if capture is active on it), otherwise it could be interface stats as we can get from OS (ifconfig stats).</div><div><br></div><div>Syvain, can you guide me a little about Gremlin module? </div><span class=""><div><span style="color:rgb(33,33,33)"><br></span></div><div><span style="color:rgb(33,33,33)">>We are going to add a `sum` step to the gremlin language for that and we could display the result in the >WebUI when pointing edges</span><span class="m_4831095358183034420inbox-inbox-Apple-converted-space" style="color:rgb(33,33,33)"> </span></div><div><span class="m_4831095358183034420inbox-inbox-Apple-converted-space" style="color:rgb(33,33,33)"><br></span></div></span><div><font color="#212121">so, Gremlin request going through "api/topology" to Analyzer; it does general logic of Metric aggregation/sum: </font><span style="color:rgb(33,33,33)">returns back synthetic data `flow` containing the result?</span></div><div><span style="color:rgb(33,33,33)"><br></span></div><div><font color="#212121">Why it is better than to get all flows stored currently in the Vertex and compute aggregation on webUI side? It might be better to offload computation from Analyzer (better for scaling).</font></div><div><font color="#212121"><br></font></div><div><font color="#212121">Thanks, Kostya</font></div><div><font color="#212121"><br></font></div><div><font color="#212121"><br></font></div><div><br></div><div><br></div></div><div class="HOEnZb"><div class="h5"><br><div class="gmail_quote"><div dir="ltr">On Wed, Nov 2, 2016 at 3:35 PM Sylvain Afchain <<a href="mailto:safchain@redhat.com" target="_blank">safchain@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello,<br class="m_4831095358183034420gmail_msg"> <br class="m_4831095358183034420gmail_msg"> So there is already a way to retrieve metrics thanks to the WebUI, not on edge but on node where the captures occur. There is an on going work to be able to aggregate metrics from multiple capture so that we could get the bandwidth on a link. We are going to add a `sum` step to the gremlin language for that and we could display the result in the WebUI when pointing edges ?<br class="m_4831095358183034420gmail_msg"> <br class="m_4831095358183034420gmail_msg"> But here I'm talking about flow capture not interface metrics. There is a opened issue[1] to bring interface metrics to the graph and I know that Mathieu wanted to work on this. What we could do is to store interface metrics as node metadata ?<br class="m_4831095358183034420gmail_msg"> <br class="m_4831095358183034420gmail_msg"> About the WebUI and updates :<br class="m_4831095358183034420gmail_msg"> <br class="m_4831095358183034420gmail_msg"> The webui is listening graph events thanks to a websocket connection. Each time there is a modification of the graph on the analyzer side, an event for the change is sent to all the websocket clients so to the webui. And as you said the event message are consumed by ProcessGraphMessage.<br class="m_4831095358183034420gmail_msg"> <br class="m_4831095358183034420gmail_msg"> Sylvain<br class="m_4831095358183034420gmail_msg"> <br class="m_4831095358183034420gmail_msg"> [1] <a href="https://github.com/skydive-project/skydive/issues/109" rel="noreferrer" class="m_4831095358183034420gmail_msg" target="_blank">https://github.com/skydive-<wbr>project/skydive/issues/109</a><br class="m_4831095358183034420gmail_msg"> <br class="m_4831095358183034420gmail_msg"> ----- Original Message -----<br class="m_4831095358183034420gmail_msg"> > Hello,<br class="m_4831095358183034420gmail_msg"> > I'm thinking about adding some metadata on webUI edges along active capture<br class="m_4831095358183034420gmail_msg"> > - bandwidth, so one can understand traffic on the given topology.<br class="m_4831095358183034420gmail_msg"> > As I see this task can be divided into the following:<br class="m_4831095358183034420gmail_msg"> > 1. Add bandwidth metadata to the Edge<br class="m_4831095358183034420gmail_msg"> > 2. When webUI gets "NewEdge" оr "NewUpdate" message in<br class="m_4831095358183034420gmail_msg"> > ProcessGraphMessage() function, need to draw this bandwidth info on the<br class="m_4831095358183034420gmail_msg"> > edge of the graph representation.<br class="m_4831095358183034420gmail_msg"> > 3. Adding new or modifying existing probe in order it will be able to<br class="m_4831095358183034420gmail_msg"> > generate/compute bandwidth data and supply it to the Agent graph.<br class="m_4831095358183034420gmail_msg"> ><br class="m_4831095358183034420gmail_msg"> > So, what do you think about the idea? Where should I start with adding such<br class="m_4831095358183034420gmail_msg"> > info?<br class="m_4831095358183034420gmail_msg"> ><br class="m_4831095358183034420gmail_msg"> > I suppose we can store the bandwidth info as metadata in these:<br class="m_4831095358183034420gmail_msg"> > 1) type graphElement struct {<br class="m_4831095358183034420gmail_msg"> > ID Identifier<br class="m_4831095358183034420gmail_msg"> > metadata Metadata<br class="m_4831095358183034420gmail_msg"> > host string<br class="m_4831095358183034420gmail_msg"> > }<br class="m_4831095358183034420gmail_msg"> > 3) type Edge struct {<br class="m_4831095358183034420gmail_msg"> > graphElement<br class="m_4831095358183034420gmail_msg"> > parent Identifier<br class="m_4831095358183034420gmail_msg"> > child Identifier<br class="m_4831095358183034420gmail_msg"> > }<br class="m_4831095358183034420gmail_msg"> > 3) type MemoryBackendEdge struct {<br class="m_4831095358183034420gmail_msg"> > *Edge<br class="m_4831095358183034420gmail_msg"> > }<br class="m_4831095358183034420gmail_msg"> ><br class="m_4831095358183034420gmail_msg"> > Another, more general question: can you please describe in two words, what<br class="m_4831095358183034420gmail_msg"> > is data flow from in memory graph of Analyzer to webUI:<br class="m_4831095358183034420gmail_msg"> > - how update of webUI started<br class="m_4831095358183034420gmail_msg"> > - when it is updated (on changes, periodically)<br class="m_4831095358183034420gmail_msg"> ><br class="m_4831095358183034420gmail_msg"> > And what flows should I look into to understand and to start playing with<br class="m_4831095358183034420gmail_msg"> > adding additional metadata described above?<br class="m_4831095358183034420gmail_msg"> ><br class="m_4831095358183034420gmail_msg"> > Thanks,<br class="m_4831095358183034420gmail_msg"> > Kostya<br class="m_4831095358183034420gmail_msg"> ><br class="m_4831095358183034420gmail_msg"> </blockquote></div> </div></div></blockquote></div><br></div>