<div dir="ltr">Hi <div class="gmail_quote"><div dir="ltr">On Mon, Aug 13, 2018 at 8:40 PM Jason Montleon <<a href="mailto:jmontleo@redhat.com">jmontleo@redhat.com</a>> wrote: </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Someone recently filed a bug for the v3.10/latest version mismatch. We have some ideas how to handle it better and are working on implementing it for 3.11 (and possibly 3.10). For now it's pretty easy to modify the dc image to get the release you want. oc patch dc -n openshift-automation-service-broker openshift-automation-service-broker --patch='{"spec":{"template":{"spec":{"containers":[{"name": "penshift-automation-service-broker", "image":"ansibleplaybookbundle/origin-ansible-service-broker:release-1.2"}]}}}}' </blockquote><div> </div><div>I tried applying the patch after my Ansible Service Broker installation was failing in the same exact way that Charles mentions, but the odd thing is that I don't have the openshift-automation-service-broker namespace. Is that to be expected or perhaps some bug? </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> release-1.0 corresponds to 3.7 release-1.1 3.9 release-1.2 3.10 latest what will become release-1.3 3.11 I'm not sure anything earlier than release-1.2 will work with the resources created by the APB so I'd stick to release-1.2 and latest if you can for now. On 08/13/2018 12:52 PM, Charles Moulliard wrote: > Is there a reason why we don't install the APB docker image tagged for > openshift 3.10 but instead latest ? > > <a href="https://github.com/openshift/openshift-ansible/blob/release-3.10/roles/ansible_service_broker/defaults/main.yml#L26" rel="noreferrer" target="_blank">https://github.com/openshift/openshift-ansible/blob/release-3.10/roles/ansible_service_broker/defaults/main.yml#L26</a> > > > > On Mon, Aug 13, 2018 at 6:42 PM, Charles Moulliard <<a href="mailto:cmoullia@redhat.com" target="_blank">cmoullia@redhat.com</a> > <mailto:<a href="mailto:cmoullia@redhat.com" target="_blank">cmoullia@redhat.com</a>>> wrote: > > FYI, events still report that the pod is unhealthy > > 6:39:49 PMWarningUnhealthy Readiness probe failed: Get > <a href="https://10.128.0.16:1338/healthz" rel="noreferrer" target="_blank">https://10.128.0.16:1338/healthz</a> <<a href="https://10.128.0.16:1338/healthz" rel="noreferrer" target="_blank">https://10.128.0.16:1338/healthz</a>>: > dial tcp <a href="http://10.128.0.16:1338" rel="noreferrer" target="_blank">10.128.0.16:1338</a> <<a href="http://10.128.0.16:1338" rel="noreferrer" target="_blank">http://10.128.0.16:1338</a>>: getsockopt: > connection refused > 4 times in the last minute > 6:39:45 PMWarningUnhealthy Liveness probe failed: Get > <a href="https://10.128.0.16:1338/healthz" rel="noreferrer" target="_blank">https://10.128.0.16:1338/healthz</a> <<a href="https://10.128.0.16:1338/healthz" rel="noreferrer" target="_blank">https://10.128.0.16:1338/healthz</a>>: > dial tcp <a href="http://10.128.0.16:1338" rel="noreferrer" target="_blank">10.128.0.16:1338</a> <<a href="http://10.128.0.16:1338" rel="noreferrer" target="_blank">http://10.128.0.16:1338</a>>: getsockopt: > connection refused > 2 times in the last > > and of course, when I browse the catalog, I don't see APB playbooks :-( > > > On Mon, Aug 13, 2018 at 6:40 PM, David Zager <<a href="mailto:dzager@redhat.com" target="_blank">dzager@redhat.com</a> > <mailto:<a href="mailto:dzager@redhat.com" target="_blank">dzager@redhat.com</a>>> wrote: > > An update to the liveness and readiness probes for 3.10 may be > appropriate in openshift-ansible. > > We are intentionally keeping the `dh-automation-broker-apb` out > of the list of services. I suspect the issue here is simply with > the log level of "not adding spec dh-automation-broker-apb to > list of services due to error transforming to service - Could > not find the parameter type for: "? > > On Mon, Aug 13, 2018 at 12:35 PM Charles Moulliard > <<a href="mailto:cmoullia@redhat.com" target="_blank">cmoullia@redhat.com</a> <mailto:<a href="mailto:cmoullia@redhat.com" target="_blank">cmoullia@redhat.com</a>>> wrote: > > To get the pod up and running, I have removed the > threshold's parameters from the APB's deployment resource > ->liveness and readinessProbes. > > But I see this error now > > time="2018-08-13T16:30:09Z" level=info msg="Broker > successfully bootstrapped on startup" > time="2018-08-13T16:30:09Z" level=info msg="Listening on > https://[::]:1338" > time="2018-08-13T16:30:09Z" level=info msg="Ansible Service > Broker Starting" > ERROR: logging before flag.Parse: I0813 16:30:09.600491 > 1 serve.go:89] Serving securely on [::]:1338 > time="2018-08-13T16:30:20Z" level=info > msg="AnsibleBroker::Catalog" > time="2018-08-13T16:30:20Z" level=error msg="not adding spec > dh-automation-broker-apb to list of services due to error > transforming to service - Could not find the parameter type > for: " > 10.128.0.1 - - [13/Aug/2018:16:30:20 +0000] "GET > /ansible-service-broker/v2/catalog HTTP/1.1" 200 273918 > time="2018-08-13T16:30:40Z" level=info > msg="AnsibleBroker::Catalog" > time="2018-08-13T16:30:40Z" level=error msg="not adding spec > dh-automation-broker-apb to list of services due to error > transforming to service - Could not find the parameter type > for: " > 10.128.0.1 - - [13/Aug/2018:16:30:40 +0000] "GET > /ansible-service-broker/v2/catalog HTTP/1.1" 200 273918 > > On Fri, Aug 10, 2018 at 8:31 AM, Georgios Andrianakis > <<a href="mailto:gandrian@redhat.com" target="_blank">gandrian@redhat.com</a> <mailto:<a href="mailto:gandrian@redhat.com" target="_blank">gandrian@redhat.com</a>>> wrote: > > Hello, > > I am seeing the same exact error as Charles under the > same conditions -> installing ASB on Openshift Origin 3.10. > > Regards > > On Thu, Aug 9, 2018 at 8:36 PM David Zager > <<a href="mailto:dzager@redhat.com" target="_blank">dzager@redhat.com</a> <mailto:<a href="mailto:dzager@redhat.com" target="_blank">dzager@redhat.com</a>>> wrote: > > After reading through the logs and seeing the way > the cut off, I'm curious if it's related to the > `initialDelaySeconds` in the broker's deployment > config > <<a href="https://github.com/openshift/openshift-ansible/blob/master/roles/ansible_service_broker/templates/asb_dc.yaml.j2#L59-L72" rel="noreferrer" target="_blank">https://github.com/openshift/openshift-ansible/blob/master/roles/ansible_service_broker/templates/asb_dc.yaml.j2#L59-L72</a>>. > With the upstream broker-apb we increased this from > 15 seconds to 120 > <<a href="https://github.com/openshift/ansible-service-broker/blob/master/apb/defaults/main.yml#L17" rel="noreferrer" target="_blank">https://github.com/openshift/ansible-service-broker/blob/master/apb/defaults/main.yml#L17</a>>. > I suspect that could be the cause. > > > On Thu, Aug 9, 2018 at 11:37 AM Jason Montleon > <<a href="mailto:jmontleo@redhat.com" target="_blank">jmontleo@redhat.com</a> <mailto:<a href="mailto:jmontleo@redhat.com" target="_blank">jmontleo@redhat.com</a>>> > wrote: > > I made some comments in irc this morning. > > Our downstream CI test are passing as of this > morning using > openshift-ansible with downstream images. From > his gist logs it looked > like the broker started, but I am not clear on > what's happening to cause > the getsockopt error. > > <a href="https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb" rel="noreferrer" target="_blank">https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb</a> > <<a href="https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb" rel="noreferrer" target="_blank">https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb</a>> > > It seems like the service name resolved, but it > couldn't connect at the > address "172.30.206.144". Perhaps there is a > misconfiguration with the > docker or openshift networking? > > On 08/09/2018 11:30 AM, John Matthews wrote: > > +Jason/David who may be able to help > > > > > > Hi Charles, > > > > "I don't see anymore the apb-etcd pod wuth > openshift 3.10, it is normal ?" > > > > Yes, this is normal for 3.10, we moved to > CRDs for the data we store, so > > we dropped our own etcd and rely on regular > k8s API. > > > > > > "Error fetching catalog. Error getting broker > catalog: > > > Gethttps://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog > > > > <<a href="https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog" rel="noreferrer" target="_blank">https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog</a> > <<a href="https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog" rel="noreferrer" target="_blank">https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog</a>>>: > > > dial tcp172.30.206.144:1338 > <<a href="http://172.30.206.144:1338/" rel="noreferrer" target="_blank">http://172.30.206.144:1338/</a>>: getsockopt: > > no route to host" > > > > Please double check the namespace the > automation broker is running in. > > I wonder if there is a bug we missed with > "openshift-ansible" deploying > > with origin. > > I believe the majority of our QE scenarios > are with openshift-ansible > > with downstream OCP or 'oc cluster up' with > origin. > > > > Guessing, but perhaps there is a mismatch > between actual namespace > > broker is in versus what we are registering > to service-catalog. > > > > > > > > > > On Thu, Aug 9, 2018 at 4:29 AM, Charles > Moulliard <<a href="mailto:cmoullia@redhat.com" target="_blank">cmoullia@redhat.com</a> > <mailto:<a href="mailto:cmoullia@redhat.com" target="_blank">cmoullia@redhat.com</a>> > > <mailto:<a href="mailto:cmoullia@redhat.com" target="_blank">cmoullia@redhat.com</a> > <mailto:<a href="mailto:cmoullia@redhat.com" target="_blank">cmoullia@redhat.com</a>>>> wrote: > > > > Additional info. The ClusterServiceBroker > reports this error > > > > Error fetching catalog. Error getting > broker catalog: Get > > > <a href="https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog" rel="noreferrer" target="_blank">https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog</a> > <<a href="https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog" rel="noreferrer" target="_blank">https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog</a>> > > > <<a href="https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog" rel="noreferrer" target="_blank">https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog</a> <<a href="https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog" rel="noreferrer" target="_blank">https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog</a>>>: > > dial tcp <a href="http://172.30.206.144:1338" rel="noreferrer" target="_blank">172.30.206.144:1338</a> > <<a href="http://172.30.206.144:1338" rel="noreferrer" target="_blank">http://172.30.206.144:1338</a>> > <<a href="http://172.30.206.144:1338" rel="noreferrer" target="_blank">http://172.30.206.144:1338</a>>: > > getsockopt: no route to host > > > > On Thu, Aug 9, 2018 at 10:22 AM, Charles > Moulliard > > <<a href="mailto:cmoullia@redhat.com" target="_blank">cmoullia@redhat.com</a> > <mailto:<a href="mailto:cmoullia@redhat.com" target="_blank">cmoullia@redhat.com</a>> > <mailto:<a href="mailto:cmoullia@redhat.com" target="_blank">cmoullia@redhat.com</a> > <mailto:<a href="mailto:cmoullia@redhat.com" target="_blank">cmoullia@redhat.com</a>>>> wrote: > > > > Hi > > > > I can't install ASB on origin 3.10 > anymore using the playbook > > > "openshift-ansible/playbooks/openshift-service-catalog/config.yml" > > > > The apb deployment starts, a pod is > created and after x minutes > > I see the status = "Failed" but the > pod's log doesn't report ERROR > > > > Log : > > > <a href="https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb" rel="noreferrer" target="_blank">https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb</a> > <<a href="https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb" rel="noreferrer" target="_blank">https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb</a>> > > > <<a href="https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb" rel="noreferrer" target="_blank">https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb</a> <<a href="https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb" rel="noreferrer" target="_blank">https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb</a>>> > > > > How can we troubleshoot and resolve > such error ? > > I don't see anymore the apb-etcd pod > wuth openshift 3.10, it is > > normal ? > > > > Regards > > > > Charles > > > > > > > > > _______________________________________________ > > Ansible-service-broker mailing list > > <a href="mailto:Ansible-service-broker@redhat.com" target="_blank">Ansible-service-broker@redhat.com</a> > <mailto:<a href="mailto:Ansible-service-broker@redhat.com" target="_blank">Ansible-service-broker@redhat.com</a>> > > <mailto:<a href="mailto:Ansible-service-broker@redhat.com" target="_blank">Ansible-service-broker@redhat.com</a> > <mailto:<a href="mailto:Ansible-service-broker@redhat.com" target="_blank">Ansible-service-broker@redhat.com</a>>> > > > <a href="https://www.redhat.com/mailman/listinfo/ansible-service-broker" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/ansible-service-broker</a> > <<a href="https://www.redhat.com/mailman/listinfo/ansible-service-broker" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/ansible-service-broker</a>> > > > <<a href="https://www.redhat.com/mailman/listinfo/ansible-service-broker" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/ansible-service-broker</a> <<a href="https://www.redhat.com/mailman/listinfo/ansible-service-broker" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/ansible-service-broker</a>>> > > > > > > -- > Jason Montleon | email: <a href="mailto:jmontleo@redhat.com" target="_blank">jmontleo@redhat.com</a> > <mailto:<a href="mailto:jmontleo@redhat.com" target="_blank">jmontleo@redhat.com</a>> > Software Engineer | gpg key: 0x069E3022 > Red Hat, Inc. | irc: jmontleo > desk: 978-392-3930 <tel:%28978%29%20392-3930> | > cell: 508-496-0663 <tel:%28508%29%20496-0663> > > _______________________________________________ > Ansible-service-broker mailing list > <a href="mailto:Ansible-service-broker@redhat.com" target="_blank">Ansible-service-broker@redhat.com</a> > <mailto:<a href="mailto:Ansible-service-broker@redhat.com" target="_blank">Ansible-service-broker@redhat.com</a>> > <a href="https://www.redhat.com/mailman/listinfo/ansible-service-broker" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/ansible-service-broker</a> > <<a href="https://www.redhat.com/mailman/listinfo/ansible-service-broker" rel="noreferrer" target="_blank">https://www.redhat.com/mailman/listinfo/ansible-service-broker</a>> > > > > -- Jason Montleon | email: <a href="mailto:jmontleo@redhat.com" target="_blank">jmontleo@redhat.com</a> Software Engineer | gpg key: 0x069E3022 Red Hat, Inc. | irc: jmontleo desk: 978-392-3930 | cell: 508-496-0663 </blockquote></div></div>