[Ansible-service-broker] pod can't start when deployed on origin 3.10

Charles Moulliard cmoullia at redhat.com
Mon Aug 13 16:52:08 UTC 2018 

Is there a reason why we don't install the APB docker image tagged for
openshift 3.10 but instead latest  ?

https://github.com/openshift/openshift-ansible/blob/release-3.10/roles/ansible_service_broker/defaults/main.yml#L26



On Mon, Aug 13, 2018 at 6:42 PM, Charles Moulliard <cmoullia at redhat.com>
wrote:

> FYI, events still report that the pod is unhealthy
>
> 6:39:49 PM Warning Unhealthy Readiness probe failed: Get
> https://10.128.0.16:1338/healthz: dial tcp 10.128.0.16:1338: getsockopt:
> connection refused
> 4 times in the last minute
> 6:39:45 PM Warning Unhealthy Liveness probe failed: Get
> https://10.128.0.16:1338/healthz: dial tcp 10.128.0.16:1338: getsockopt:
> connection refused
> 2 times in the last
>
> and of course, when I browse the catalog, I don't see APB playbooks :-(
>
>
> On Mon, Aug 13, 2018 at 6:40 PM, David Zager <dzager at redhat.com> wrote:
>
>> An update to the liveness and readiness probes for 3.10 may be
>> appropriate in openshift-ansible.
>>
>> We are intentionally keeping the `dh-automation-broker-apb` out of the
>> list of services. I suspect the issue here is simply with the log level of "not
>> adding spec dh-automation-broker-apb to list of services due to error
>> transforming to service - Could not find the parameter type for: "?
>>
>> On Mon, Aug 13, 2018 at 12:35 PM Charles Moulliard <cmoullia at redhat.com>
>> wrote:
>>
>>> To get the pod up and running, I have removed the threshold's parameters
>>> from the APB's deployment resource ->liveness and readinessProbes.
>>>
>>> But I see this error now
>>>
>>> time="2018-08-13T16:30:09Z" level=info msg="Broker successfully
>>> bootstrapped on startup"
>>> time="2018-08-13T16:30:09Z" level=info msg="Listening on https://
>>> [::]:1338"
>>> time="2018-08-13T16:30:09Z" level=info msg="Ansible Service Broker
>>> Starting"
>>> ERROR: logging before flag.Parse: I0813 16:30:09.600491       1
>>> serve.go:89] Serving securely on [::]:1338
>>> time="2018-08-13T16:30:20Z" level=info msg="AnsibleBroker::Catalog"
>>> time="2018-08-13T16:30:20Z" level=error msg="not adding spec
>>> dh-automation-broker-apb to list of services due to error transforming to
>>> service - Could not find the parameter type for: "
>>> 10.128.0.1 - - [13/Aug/2018:16:30:20 +0000] "GET
>>> /ansible-service-broker/v2/catalog HTTP/1.1" 200 273918
>>> time="2018-08-13T16:30:40Z" level=info msg="AnsibleBroker::Catalog"
>>> time="2018-08-13T16:30:40Z" level=error msg="not adding spec
>>> dh-automation-broker-apb to list of services due to error transforming to
>>> service - Could not find the parameter type for: "
>>> 10.128.0.1 - - [13/Aug/2018:16:30:40 +0000] "GET
>>> /ansible-service-broker/v2/catalog HTTP/1.1" 200 273918
>>>
>>> On Fri, Aug 10, 2018 at 8:31 AM, Georgios Andrianakis <
>>> gandrian at redhat.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> I am seeing the same exact error as Charles under the same conditions
>>>> -> installing ASB on Openshift Origin 3.10.
>>>>
>>>> Regards
>>>>
>>>> On Thu, Aug 9, 2018 at 8:36 PM David Zager <dzager at redhat.com> wrote:
>>>>
>>>>> After reading through the logs and seeing the way the cut off, I'm
>>>>> curious if it's related to the `initialDelaySeconds` in the broker's
>>>>> deployment config
>>>>> <https://github.com/openshift/openshift-ansible/blob/master/roles/ansible_service_broker/templates/asb_dc.yaml.j2#L59-L72>.
>>>>> With the upstream broker-apb we increased this from 15 seconds to 120
>>>>> <https://github.com/openshift/ansible-service-broker/blob/master/apb/defaults/main.yml#L17>.
>>>>> I suspect that could be the cause.
>>>>>
>>>>>
>>>>> On Thu, Aug 9, 2018 at 11:37 AM Jason Montleon <jmontleo at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> I made some comments in irc this morning.
>>>>>>
>>>>>> Our downstream CI test are passing as of this morning using
>>>>>> openshift-ansible with downstream images. From his gist logs it
>>>>>> looked
>>>>>> like the broker started, but I am not clear on what's happening to
>>>>>> cause
>>>>>> the getsockopt error.
>>>>>>
>>>>>> https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a30e3ddcb
>>>>>>
>>>>>> It seems like the service name resolved, but it couldn't connect at
>>>>>> the
>>>>>> address "172.30.206.144". Perhaps there is a misconfiguration with
>>>>>> the
>>>>>> docker or openshift networking?
>>>>>>
>>>>>> On 08/09/2018 11:30 AM, John Matthews wrote:
>>>>>> > +Jason/David who may be able to help
>>>>>> >
>>>>>> >
>>>>>> > Hi Charles,
>>>>>> >
>>>>>> > "I don't see anymore the apb-etcd pod wuth openshift 3.10, it is
>>>>>> normal ?"
>>>>>> >
>>>>>> > Yes, this is normal for 3.10, we moved to CRDs for the data we
>>>>>> store, so
>>>>>> > we dropped our own etcd and rely on regular k8s API.
>>>>>> >
>>>>>> >
>>>>>> > "Error fetching catalog. Error getting broker catalog:
>>>>>> > Gethttps://asb.openshift-ansible-service-broker.svc:1338/
>>>>>> ansible-service-broker/v2/catalog
>>>>>> > <https://asb.openshift-ansible-service-broker.svc:1338/
>>>>>> ansible-service-broker/v2/catalog>:
>>>>>> > dial tcp172.30.206.144:1338 <http://172.30.206.144:1338/>:
>>>>>> getsockopt:
>>>>>> > no route to host"
>>>>>> >
>>>>>> > Please double check the namespace the automation broker is running
>>>>>> in.
>>>>>> > I wonder if there is a bug we missed with "openshift-ansible"
>>>>>> deploying
>>>>>> > with origin.
>>>>>> > I believe the majority of our QE scenarios are with
>>>>>> openshift-ansible
>>>>>> > with downstream OCP or 'oc cluster up' with origin.
>>>>>> >
>>>>>> > Guessing, but perhaps there is a mismatch between actual namespace
>>>>>> > broker is in versus what we are registering to service-catalog.
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > On Thu, Aug 9, 2018 at 4:29 AM, Charles Moulliard <
>>>>>> cmoullia at redhat.com
>>>>>> > <mailto:cmoullia at redhat.com>> wrote:
>>>>>> >
>>>>>> >     Additional info. The ClusterServiceBroker reports this error
>>>>>> >
>>>>>> >     Error fetching catalog. Error getting broker catalog: Get
>>>>>> >     https://asb.openshift-ansible-service-broker.svc:1338/
>>>>>> ansible-service-broker/v2/catalog
>>>>>> >     <https://asb.openshift-ansible-service-broker.svc:1338/
>>>>>> ansible-service-broker/v2/catalog>:
>>>>>> >     dial tcp 172.30.206.144:1338 <http://172.30.206.144:1338>:
>>>>>> >     getsockopt: no route to host
>>>>>> >
>>>>>> >     On Thu, Aug 9, 2018 at 10:22 AM, Charles Moulliard
>>>>>> >     <cmoullia at redhat.com <mailto:cmoullia at redhat.com>> wrote:
>>>>>> >
>>>>>> >         Hi
>>>>>> >
>>>>>> >         I can't install ASB on origin 3.10 anymore using the
>>>>>> playbook
>>>>>> >         "openshift-ansible/playbooks/openshift-service-catalog/conf
>>>>>> ig.yml"
>>>>>> >
>>>>>> >         The apb deployment starts, a pod is created and after x
>>>>>> minutes
>>>>>> >         I see the status = "Failed" but the pod's log doesn't
>>>>>> report ERROR
>>>>>> >
>>>>>> >         Log :
>>>>>> >         https://gist.github.com/cmoulliard/716468a9e9beb21ba860806a
>>>>>> 30e3ddcb
>>>>>> >         <https://gist.github.com/cmoulliard/716468a9e9beb21ba860806
>>>>>> a30e3ddcb>
>>>>>> >
>>>>>> >         How can we troubleshoot and resolve such error ?
>>>>>> >         I don't see anymore the apb-etcd pod wuth openshift 3.10,
>>>>>> it is
>>>>>> >         normal ?
>>>>>> >
>>>>>> >         Regards
>>>>>> >
>>>>>> >         Charles
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >     _______________________________________________
>>>>>> >     Ansible-service-broker mailing list
>>>>>> >     Ansible-service-broker at redhat.com
>>>>>> >     <mailto:Ansible-service-broker at redhat.com>
>>>>>> >     https://www.redhat.com/mailman/listinfo/ansible-service-broker
>>>>>> >     <https://www.redhat.com/mailman/listinfo/ansible-service-broker
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>>
>>>>>> --
>>>>>> Jason Montleon     | email: jmontleo at redhat.com
>>>>>> Software Engineer  | gpg key: 0x069E3022
>>>>>> Red Hat, Inc.      | irc: jmontleo
>>>>>> desk: 978-392-3930 <(978)%20392-3930> | cell: 508-496-0663
>>>>>> <(508)%20496-0663>
>>>>>>
>>>>> _______________________________________________
>>>>> Ansible-service-broker mailing list
>>>>> Ansible-service-broker at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/ansible-service-broker
>>>>>
>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/ansible-service-broker/attachments/20180813/b81c6c5a/attachment.htm>

More information about the Ansible-service-broker mailing list