[Ansible-service-broker] pod can't start when deployed on origin 3.10
Jason Montleon
jmontleo at redhat.com
Mon Aug 13 17:40:19 UTC 2018
Someone recently filed a bug for the v3.10/latest version mismatch. We
have some ideas how to handle it better and are working on implementing
it for 3.11 (and possibly 3.10).
For now it's pretty easy to modify the dc image to get the release you want.
oc patch dc -n openshift-automation-service-broker
openshift-automation-service-broker
--patch='{"spec":{"template":{"spec":{"containers":[{"name":
"penshift-automation-service-broker",
"image":"ansibleplaybookbundle/origin-ansible-service-broker:release-1.2"}]}}}}'
release-1.0 corresponds to 3.7
release-1.1 3.9
release-1.2 3.10
latest what will become release-1.3 3.11
I'm not sure anything earlier than release-1.2 will work with the
resources created by the APB so I'd stick to release-1.2 and latest if
you can for now.
On 08/13/2018 12:52 PM, Charles Moulliard wrote:
> Is there a reason why we don't install the APB docker image tagged for
> openshift 3.10 but instead latest ?
>
> https://github.com/openshift/openshift-ansible/blob/release-3.10/roles/ansible_service_broker/defaults/main.yml#L26
>
>
>
> On Mon, Aug 13, 2018 at 6:42 PM, Charles Moulliard <cmoullia at redhat.com
> <mailto:cmoullia at redhat.com>> wrote:
>
> FYI, events still report that the pod is unhealthy
>
> 6:39:49 PMWarningUnhealthy Readiness probe failed: Get
> https://10.128.0.16:1338/healthz <https://10.128.0.16:1338/healthz>:
> dial tcp 10.128.0.16:1338 <http://10.128.0.16:1338>: getsockopt:
> connection refused
> 4 times in the last minute
> 6:39:45 PMWarningUnhealthy Liveness probe failed: Get
> https://10.128.0.16:1338/healthz <https://10.128.0.16:1338/healthz>:
> dial tcp 10.128.0.16:1338 <http://10.128.0.16:1338>: getsockopt:
> connection refused
> 2 times in the last
>
> and of course, when I browse the catalog, I don't see APB playbooks :-(
>
>
> On Mon, Aug 13, 2018 at 6:40 PM, David Zager <dzager at redhat.com
> <mailto:dzager at redhat.com>> wrote:
>
> An update to the liveness and readiness probes for 3.10 may be
> appropriate in openshift-ansible.
>
> We are intentionally keeping the `dh-automation-broker-apb` out
> of the list of services. I suspect the issue here is simply with
> the log level of "not adding spec dh-automation-broker-apb to
> list of services due to error transforming to service - Could
> not find the parameter type for: "?
>
> On Mon, Aug 13, 2018 at 12:35 PM Charles Moulliard
> <cmoullia at redhat.com <mailto:cmoullia at redhat.com>> wrote:
>
> To get the pod up and running, I have removed the
> threshold's parameters from the APB's deployment resource
> ->liveness and readinessProbes.
>
> But I see this error now
>
> time="2018-08-13T16:30:09Z" level=info msg="Broker
> successfully bootstrapped on startup"
> time="2018-08-13T16:30:09Z" level=info msg="Listening on
> https://[::]:1338"
> time="2018-08-13T16:30:09Z" level=info msg="Ansible Service
> Broker Starting"
> ERROR: logging before flag.Parse: I0813 16:30:09.600491
> 1 serve.go:89] Serving securely on [::]:1338
> time="2018-08-13T16:30:20Z" level=info
> msg="AnsibleBroker::Catalog"
> time="2018-08-13T16:30:20Z" level=error msg="not adding spec
> dh-automation-broker-apb to list of services due to error
> transforming to service - Could not find the parameter type
> for: "
> 10.128.0.1 - - [13/Aug/2018:16:30:20 +0000] "GET
> /ansible-service-broker/v2/catalog HTTP/1.1" 200 273918
> time="2018-08-13T16:30:40Z" level=info
> msg="AnsibleBroker::Catalog"
> time="2018-08-13T16:30:40Z" level=error msg="not adding spec
> dh-automation-broker-apb to list of services due to error
> transforming to service - Could not find the parameter type
> for: "
> 10.128.0.1 - - [13/Aug/2018:16:30:40 +0000] "GET
> /ansible-service-broker/v2/catalog HTTP/1.1" 200 273918
>
> On Fri, Aug 10, 2018 at 8:31 AM, Georgios Andrianakis
> <gandrian at redhat.com <mailto:gandrian at redhat.com>> wrote:
>
> Hello,
>
> I am seeing the same exact error as Charles under the
> same conditions -> installing ASB on Openshift Origin 3.10.
>
> Regards
>
> On Thu, Aug 9, 2018 at 8:36 PM David Zager
> <dzager at redhat.com <mailto:dzager at redhat.com>> wrote:
>
> After reading through the logs and seeing the way
> the cut off, I'm curious if it's related to the
> `initialDelaySeconds` in the broker's deployment
> config
> <https://github.com/openshift/openshift-ansible/blob/master/roles/ansible_service_broker/templates/asb_dc.yaml.j2#L59-L72>.
> With the upstream broker-apb we increased this from
> 15 seconds to 120
> <https://github.com/openshift/ansible-service-broker/blob/master/apb/defaults/main.yml#L17>.
> I suspect that could be the cause.
>
>
> On Thu, Aug 9, 2018 at 11:37 AM Jason Montleon
> <jmontleo at redhat.com <mailto:jmontleo at redhat.com>>
> wrote:
>
> I made some comments in irc this morning.
>
> Our downstream CI test are passing as of this
> morning using
> openshift-ansible with downstream images. From
> his gist logs it looked
> like the broker started, but I am not clear on
> what's happening to cause
> the getsockopt error.
>
> https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb
> <https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb>
>
> It seems like the service name resolved, but it
> couldn't connect at the
> address "172.30.206.144". Perhaps there is a
> misconfiguration with the
> docker or openshift networking?
>
> On 08/09/2018 11:30 AM, John Matthews wrote:
> > +Jason/David who may be able to help
> >
> >
> > Hi Charles,
> >
> > "I don't see anymore the apb-etcd pod wuth
> openshift 3.10, it is normal ?"
> >
> > Yes, this is normal for 3.10, we moved to
> CRDs for the data we store, so
> > we dropped our own etcd and rely on regular
> k8s API.
> >
> >
> > "Error fetching catalog. Error getting broker
> catalog:
> >
> Gethttps://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog
>
> >
> <https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog
> <https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog>>:
>
> > dial tcp172.30.206.144:1338
> <http://172.30.206.144:1338/>: getsockopt:
> > no route to host"
> >
> > Please double check the namespace the
> automation broker is running in.
> > I wonder if there is a bug we missed with
> "openshift-ansible" deploying
> > with origin.
> > I believe the majority of our QE scenarios
> are with openshift-ansible
> > with downstream OCP or 'oc cluster up' with
> origin.
> >
> > Guessing, but perhaps there is a mismatch
> between actual namespace
> > broker is in versus what we are registering
> to service-catalog.
> >
> >
> >
> >
> > On Thu, Aug 9, 2018 at 4:29 AM, Charles
> Moulliard <cmoullia at redhat.com
> <mailto:cmoullia at redhat.com>
> > <mailto:cmoullia at redhat.com
> <mailto:cmoullia at redhat.com>>> wrote:
> >
> > Additional info. The ClusterServiceBroker
> reports this error
> >
> > Error fetching catalog. Error getting
> broker catalog: Get
> >
> https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog
> <https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog>
> >
> <https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog <https://asb.openshift-ansible-service-broker.svc:1338/ansible-service-broker/v2/catalog>>:
> > dial tcp 172.30.206.144:1338
> <http://172.30.206.144:1338>
> <http://172.30.206.144:1338>:
> > getsockopt: no route to host
> >
> > On Thu, Aug 9, 2018 at 10:22 AM, Charles
> Moulliard
> > <cmoullia at redhat.com
> <mailto:cmoullia at redhat.com>
> <mailto:cmoullia at redhat.com
> <mailto:cmoullia at redhat.com>>> wrote:
> >
> > Hi
> >
> > I can't install ASB on origin 3.10
> anymore using the playbook
> >
> "openshift-ansible/playbooks/openshift-service-catalog/config.yml"
> >
> > The apb deployment starts, a pod is
> created and after x minutes
> > I see the status = "Failed" but the
> pod's log doesn't report ERROR
> >
> > Log :
> >
> https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb
> <https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb>
> >
> <https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb <https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb>>
> >
> > How can we troubleshoot and resolve
> such error ?
> > I don't see anymore the apb-etcd pod
> wuth openshift 3.10, it is
> > normal ?
> >
> > Regards
> >
> > Charles
> >
> >
> >
> >
> _______________________________________________
> > Ansible-service-broker mailing list
> > Ansible-service-broker at redhat.com
> <mailto:Ansible-service-broker at redhat.com>
> > <mailto:Ansible-service-broker at redhat.com
> <mailto:Ansible-service-broker at redhat.com>>
> >
> https://www.redhat.com/mailman/listinfo/ansible-service-broker
> <https://www.redhat.com/mailman/listinfo/ansible-service-broker>
> >
> <https://www.redhat.com/mailman/listinfo/ansible-service-broker <https://www.redhat.com/mailman/listinfo/ansible-service-broker>>
> >
> >
>
> --
> Jason Montleon | email: jmontleo at redhat.com
> <mailto:jmontleo at redhat.com>
> Software Engineer | gpg key: 0x069E3022
> Red Hat, Inc. | irc: jmontleo
> desk: 978-392-3930 <tel:%28978%29%20392-3930> |
> cell: 508-496-0663 <tel:%28508%29%20496-0663>
>
> _______________________________________________
> Ansible-service-broker mailing list
> Ansible-service-broker at redhat.com
> <mailto:Ansible-service-broker at redhat.com>
> https://www.redhat.com/mailman/listinfo/ansible-service-broker
> <https://www.redhat.com/mailman/listinfo/ansible-service-broker>
>
>
>
>
--
Jason Montleon | email: jmontleo at redhat.com
Software Engineer | gpg key: 0x069E3022
Red Hat, Inc. | irc: jmontleo
desk: 978-392-3930 | cell: 508-496-0663
More information about the Ansible-service-broker
mailing list