<div dir="ltr">Thanks for your responses Sylvain. I've tried with a new query using your suggestion and it works really good.<div> </div><div>G.V().Has('Name', Regex('_openshift-infra_')).Out().Has("Type", "veth")</div><div> </div><div>I'm not using IRC anymore as we always have issues to connect to Red Hat VPN from Customers. Are you guys in RocketChat or Slack?</div><div> </div><div>Regarding "sensibility" what I mean is that every time we have tried to enable agents on each Cluster Node, we were not able to see them in Skydive and we got errors like the one I shared with you last week:</div><div> </div><div>wsserver.go:194 http (*WSClient).readPump > ERRO 015 Error while reading websocket from : websocket: close 1001</div><div> </div><div>We have also noticed that if we disabled the agent for a particular Node and we enabled again later, the Node doesn't appear anymore and we have to restart the analyzer and elasticsearch containers. These are ephemeral yet, so restarting them basically removes every collected data and we can start again.</div><div> </div><div>Thanks,</div><div>Mak <div> </div></div></div><div class="gmail_extra"> <div class="gmail_quote">On Wed, Mar 1, 2017 at 11:19 AM, Sylvain Afchain <<a href="mailto:safchain@redhat.com" target="_blank">safchain@redhat.com</a>> wrote: <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Marcos, According to your screenshots, you wrote a gremlin expression that matches namespaces. Skydive can't start a capture on such object. If you want to start a capture on all the interfaces(eth0) within a namespaces you need to add the "Out()" step. Ex: G.V().Has('Name', Regex('_test123_')).Out().Has("Name", "eth0") Another option is to add captures on ovs bridges so that you will capture all the traffic. Once started you should be able to filter the traffic for the namespace interfaces like this. G.V().Has('Name', Regex('_test123_')).Out().Flows() Meaning: G.V().Has('Name', Regex('_test123_')).Out() will return you the interfaces belonging to namespaces. Adding the Flows() step will return flows of the namespace interfaces. What do you mean by "sensible" and "sync" ? There is an heartbeat mechanism between agents and analyzers which can be adapted in the conf: ws_pong_timeout. The default value is 5 sec which is usually enough but can be increased. We did some scaling test and we never add such issue between agents and analyzers. Feel free to join the skydive-project chan on IRC. Thanks <div class="HOEnZb"><div class="h5"> Sylvain ----- Mail original ----- > Hi Sylvain, > > What I mean is that if I use the Regex method for the capture, Skydive > filter the objects which belong to that Namespace correctly as you can see > in [skydive01.png] and [skydive02.png], but no interfaces are included so > there are no flows as you can see in [skydive03.png]. But if I select one > of the 'veth' interfaces corresponding to these containers I can see the > flow [skydive04.png] > > From the tests I have been performed and from some Customers feedback, we > also have the feeling that the communication between the agents and the > analyser is quite "sensible", and we got many warnings when the sync > duration exceeds 1s. Is this something configurable to be more "permissive"? > > Thanks, > Mak > > > On Tue, Feb 28, 2017 at 3:33 PM, Sylvain Afchain <<a href="mailto:safchain@redhat.com">safchain@redhat.com</a>> > wrote: > > > Hi Marcos, > > > > Where did you start your capture ? Do you have flows ? I mean just doing > > "G.Flows()", if so can you check that ANodeTID or BNodeTID match a node TID > > of the namespaces ? > > > > Thanks, > > > > Sylvain > > > > ----- Mail original ----- > > > Hi Sylvain, > > > > > > It works almost perfect, thanks. I can see the pods for the namespace, > > but > > > I can't see any flows... > > > > > > Regards, > > > Mak > > > > > > On Thu, Feb 23, 2017 at 3:05 PM, Sylvain Afchain <<a href="mailto:safchain@redhat.com">safchain@redhat.com</a>> > > > wrote: > > > > > > > Hi Marcos, > > > > > > > > From what I understand, in OCP there is one network namespace per Pod, > > > > each Pod belongs to a project. The name of the project is contained in > > the > > > > "Name" of the Skydive nodes. Let's say we have a project called > > "test123", > > > > we will have > > > > nodes(network namespaces) with names like "k8s_..._test123_...". So if > > you > > > > do the following request, you should be able to get all the network > > > > namespaces (meaning pods) for a project. > > > > > > > > G.V().Has('Name', Regex('_test123_')) > > > > > > > > Then if you want to get flows for these pods, just add > > > > > > > > G.V().Has('Name', Regex('_test123_')).Out().Flows() > > > > > > > > The regex stuff is not ideal, we are working to add the project name to > > > > the node metadata (<a href="https://softwarefactory-project.io/r/6673" rel="noreferrer" target="_blank">https://softwarefactory-project.io/r/6673</a>). > > > > > > > > Tell me if it helps a bit :) > > > > > > > > Thanks, > > > > > > > > Sylvain > > > > > > > > ----- Mail original ----- > > > > > Hi Marcos, > > > > > > > > > > Great that you fixed the issue, feel free to open an issue if you > > think > > > > it's > > > > > needed. > > > > > > > > > > For the topology I will inject it tomorrow and I will send you a > > bunch of > > > > > questions accordingly :) > > > > > > > > > > Regards, > > > > > > > > > > Sylvain > > > > > > > > > > ----- Mail original ----- > > > > > > Hi Sylvain, > > > > > > > > > > > > The problem has been solved ussing the kubernetes yaml file [1] > > > > (adding the > > > > > > ovs plugin) for the DaemonSet definition instead the openshift one > > [2]. > > > > > > > > > > > > [1] > > > > > > <a href="https://github.com/skydive-project/skydive/blob/master/" rel="noreferrer" target="_blank">https://github.com/skydive-project/skydive/blob/master/</a> > > > > contrib/kubernetes/skydive.yaml > > > > > > [2] > > > > > > <a href="https://github.com/skydive-project/skydive/blob/master/" rel="noreferrer" target="_blank">https://github.com/skydive-project/skydive/blob/master/</a> > > > > contrib/openshift/skydive-template.yaml > > > > > > > > > > > > I've also enclosed the topology file. > > > > > > > > > > > > Thanks, > > > > > > Mak > > > > > > > > > > > > On Wed, Feb 22, 2017 at 12:14 PM, Marcos Entenza Garcia < > > > > <a href="mailto:mak@redhat.com">mak@redhat.com</a>> > > > > > > wrote: > > > > > > > > > > > > > Hi Sylvain, > > > > > > > > > > > > > > Yes, it's blocking now as we don't see the topology. As you said, > > > > this > > > > > > > error appears every time we try to view the topology through the > > web > > > > ui, > > > > > > > and also the API is retunring just null values. > > > > > > > > > > > > > > {"Nodes":[],"Edges":[]} > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Mak > > > > > > > > > > > > > > > > > > > > > On Wed, Feb 22, 2017 at 11:26 AM, Sylvain Afchain < > > > > <a href="mailto:safchain@redhat.com">safchain@redhat.com</a>> > > > > > > > wrote: > > > > > > > > > > > > > >> About the error, is it blocking ? because this kind of message > > > > appears > > > > > > >> when there is a websocket disconnection which can be between > > > > > > >> analyzer/agent > > > > > > >> or just between analyzer/webui following a browser refresh. > > > > > > >> > > > > > > >> For the VNID, to be sure to understand your deployment can you > > send > > > > us > > > > > > >> the topology "dump" so that we could re-inject it to an > > analyzer on > > > > our > > > > > > >> side. > > > > > > >> > > > > > > >> A "curl" to http://<analyzer>/api/topology will give you the > > > > "dump". > > > > > > >> > > > > > > >> Thanks, > > > > > > >> > > > > > > >> Sylvain > > > > > > >> > > > > > > >> ----- Mail original ----- > > > > > > >> > Hi guys, > > > > > > >> > > > > > > > >> > Thanks for your responses. > > > > > > >> > > > > > > > >> > @Sylvain, I understand your approach but normally Nodes are > > shared > > > > > > >> between > > > > > > >> > different projects in OCP, so the only way to identify the > > > > specific > > > > > > >> traffic > > > > > > >> > for a Namespace is filtering but the VNID, but correct me if > > I'm > > > > > > >> > wrong. > > > > > > >> > > > > > > > >> > We are now getting the same error on different deployments > > for the > > > > > > >> > skydive-analyzer image: > > > > > > >> > > > > > > > >> > wsserver.go:194 http (*WSClient).readPump > ERRO 015 Error > > while > > > > > > >> > reading > > > > > > >> > websocket from : websocket: close 1001 > > > > > > >> > > > > > > > >> > Have you guys git an idea what can cause this? > > > > > > >> > > > > > > > >> > Thanks, > > > > > > >> > Mak > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > On Wed, Feb 22, 2017 at 8:47 AM, Sylvain Afchain < > > > > <a href="mailto:safchain@redhat.com">safchain@redhat.com</a>> > > > > > > >> > wrote: > > > > > > >> > > > > > > > >> > > Hi Marcos, > > > > > > >> > > > > > > > > >> > > Beyond the VNI support that Nicolas explained, there is a > > way > > > > to get > > > > > > >> flows > > > > > > >> > > for a given namespace. I guess you started a capture at the > > ovs > > > > > > >> bridge, if > > > > > > >> > > so you can do the following gremlin request: > > > > > > >> > > > > > > > > >> > > G.V().Has('Name', 'ns1').Out().Flows() > > > > > > >> > > > > > > > > >> > > which has to be read like this: get all the nodes with the > > name > > > > > > >> > > 'ns1' > > > > > > >> > > (your namespace), then returns the nodes belonging to this > > > > > > >> > > namespace, > > > > > > >> > > finally returns the flows for those interfaces. > > > > > > >> > > > > > > > > >> > > Regards, > > > > > > >> > > > > > > > > >> > > Sylvain > > > > > > >> > > > > > > > > >> > > ----- Mail original ----- > > > > > > >> > > > Hi Marcos, > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > For the moment it's not possible to filter out the Flows > > with > > > > a > > > > > > >> specific > > > > > > >> > > VNI, > > > > > > >> > > > but a patch [1] is underway to support that by adding a > > > > network.ID > > > > > > >> on > > > > > > >> > > each > > > > > > >> > > > UUID where ID is GRE.key or Geneve.ID or VXLAN.VNI and > > > > Link.ID for > > > > > > >> > > <a href="http://VLAN.ID" rel="noreferrer" target="_blank">VLAN.ID</a> > > > > > > >> > > > > > > > > > >> > > > So it would be possible to filter out all flows with a > > kind of > > > > > > >> query like > > > > > > >> > > > that : > > > > > > >> > > > G.V().Flows().Has(parentUUID, Within( > > > > > > >> > > > G.V().Flows().Has(application, > > > > > > >> > > 'VXLAN', > > > > > > >> > > > network.ID,<target_VNI>).Dedup() )) > > > > > > >> > > > > > > > > > >> > > > But keep it mind it's better practice to start only > > capture > > > > > > >> interfaces > > > > > > >> > > of a > > > > > > >> > > > specific namespace, for example : > > > > > > >> > > > G.V().Has('Type', 'netns', 'Name', > > 'vm1').Out().Has('State', > > > > 'UP') > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > Regards, > > > > > > >> > > > > > > > > > >> > > > Nicolas > > > > > > >> > > > > > > > > > >> > > > [1] <a href="https://softwarefactory-project.io/r/#/c/6206/" rel="noreferrer" target="_blank">https://softwarefactory-project.io/r/#/c/6206/</a> > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > On Wed, Feb 22, 2017 at 3:22 AM, Marcos Entenza Garcia < > > > > > > >> <a href="mailto:mak@redhat.com">mak@redhat.com</a> > > > > > > >> > > > > > > > > > >> > > > wrote: > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > All, > > > > > > >> > > > > > > > > > >> > > > I've got a Skydive deployment in an OCP ovs-multitenat > > > > Cluster and > > > > > > >> I was > > > > > > >> > > > wondering if there is any method I could use to filter the > > > > traffic > > > > > > >> for > > > > > > >> > > just > > > > > > >> > > > one of the namespaces as all of them has different VNID. > > The > > > > flow > > > > > > >> schema > > > > > > >> > > > doesn't seem to support that and can't find that filter > > to be > > > > > > >> > > > added > > > > > > >> to a > > > > > > >> > > > Gremlin query. > > > > > > >> > > > > > > > > > >> > > > Any ideas? > > > > > > >> > > > > > > > > > >> > > > Thanks, > > > > > > >> > > > Mak > > > > > > >> > > > > > > > > > >> > > > _______________________________________________ > > > > > > >> > > > Skydive-dev mailing list > > > > > > >> > > > <a href="mailto:Skydive-dev@redhat.com">Skydive-dev@redhat.com</a> > > > > > > >> > > > <a href="https://www.redhat.com/mailman/listinfo/skydive-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/skydive-dev</a> > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > _______________________________________________ > > > > > > >> > > > Skydive-dev mailing list > > > > > > >> > > > <a href="mailto:Skydive-dev@redhat.com">Skydive-dev@redhat.com</a> > > > > > > >> > > > <a href="https://www.redhat.com/mailman/listinfo/skydive-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/skydive-dev</a> > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > > > Skydive-dev mailing list > > > > > <a href="mailto:Skydive-dev@redhat.com">Skydive-dev@redhat.com</a> > > > > > <a href="https://www.redhat.com/mailman/listinfo/skydive-dev" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/skydive-dev</a> > > > > > > > > > > > > > > > </div></div></blockquote></div> </div>