[EnMasse] EnMasse 0.12.0: Addresses not getting ready
Ulf Lilleengen
lulf at redhat.com
Tue Aug 8 09:32:18 UTC 2017
Hi Carsten,
Thanks for the detailed bug report. After looking at the code, I see
that the exception is what is preventing the status to get updated.
I'll try to reproduce this as the tls entry should have been added
afaict from looking at the code. I will apply a fix, and tag a 0.12.1.
Sorry for the inconvenience!
Best regards,
Ulf
On 08. aug. 2017 10:20, Lohmann Carsten (INST/ECS4) wrote:
> Hi,
>
> we've done an upgrade to EnMasse 0.12.0 and are now having the issue,
> that addresses registered via the address controller REST interface are
> not getting ready ("isReady" stays false).
>
> Test was done on a local minikube cluster, using the
> kubernetes/enmasse.yaml deployment file.
>
> When looking at the log of the address-controller, there is this
> exception (occurring repeatedly):
>
> ---
>
> 2017-08-07 14:53:32 WARN WatcherVerticle:92 - Exception doing resource
> update
>
> java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
>
> at java.util.ArrayList.rangeCheck(ArrayList.java:653)
>
> at java.util.ArrayList.get(ArrayList.java:429)
>
> at
> enmasse.controller.standard.StandardController.ingressToEndpoint(StandardController.java:191)
>
> at
> java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
>
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
>
> at
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
>
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
>
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
>
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
>
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
>
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
>
> at
> enmasse.controller.standard.StandardController.updateEndpoints(StandardController.java:158)
>
> at
> enmasse.controller.standard.StandardController.resourcesUpdated(StandardController.java:93)
>
> at
> enmasse.controller.common.WatcherVerticle.run(WatcherVerticle.java:89)
>
> at java.lang.Thread.run(Thread.java:748)
>
> ---
>
> Looking at StandardController.ingressToEndpoint(), line 191 is:
>
> builder.setHost(ingress.getSpec().getTls().get(0).getHosts().get(0));
>
> I guess the ingress here is the "messaging" ingress (it has a
> "io.enmasse.certSecret" annotation set).
>
> It looks like this:
>
> ---
>
> {
>
> "apiVersion": "extensions/v1beta1",
>
> "kind": "Ingress",
>
> "metadata": {
>
> "annotations": {
>
> "addressSpace": "default",
>
> "io.enmasse.certSecret": "external-certs-messaging"
>
> },
>
> "creationTimestamp": "2017-08-07T14:53:31Z",
>
> "generation": 1,
>
> "name": "messaging",
>
> "namespace": "hono",
>
> "resourceVersion": "509798",
>
> "selfLink":
> "/apis/extensions/v1beta1/namespaces/hono/ingresses/messaging",
>
> "uid": "2de5c2fb-7b80-11e7-adee-080027c98449"
>
> },
>
> "spec": {
>
> "backend": {
>
> "serviceName": "messaging",
>
> "servicePort": "amqps"
>
> }
>
> },
>
> "status": {
>
> "loadBalancer": {}
>
> }
>
> }
>
> ---
>
> So, no "tls" below "spec".
>
> Here is some more log output:
>
> ragent:
>
> ----
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info Router connected from
> Router.qdrouterd-1128330706-25p37
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info EXTRACT, results: ,
> attributes:
> name,identity,type,prefix,distribution,waypoint,ingressPhase,egressPhase
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info retrieved addresses for
> Router.qdrouterd-1128330706-25p37: {}
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info retrieved link routes for
> Router.qdrouterd-1128330706-25p37: raw->
> [{"name":"override.lwt_in","identity":"1","type":"org.apache.qpid.dispatch.router.config.linkRoute","prefix":"$lwt","distribution":"linkBalanced","connection":null,"containerId":"lwt-service","dir":"in","operStatus":"active"},{"name":"override.lwt_out","identity":"2","type":"org.apache.qpid.dispatch.router.config.linkRoute","prefix":"$lwt","distribution":"linkBalanced","connection":null,"containerId":"lwt-service","dir":"out","operStatus":"active"},{"name":"override.locate","identity":"3","type":"org.apache.qpid.dispatch.router.config.linkRoute","prefix":"locate","distribution":"linkBalanced","connection":"subscription-service","containerId":null,"dir":"out","operStatus":"active"}]
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info retrieved link routes for
> Router.qdrouterd-1128330706-25p37: {}
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info updating addresses for
> Router.qdrouterd-1128330706-25p37
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info defining address
> telemetry/DEFAULT_TENANT on router Router.qdrouterd-1128330706-25p37
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info defining in link route
> event/DEFAULT_TENANT on router Router.qdrouterd-1128330706-25p37
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info defining out link route
> event/DEFAULT_TENANT on router Router.qdrouterd-1128330706-25p37
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info addresses updated for
> Router.qdrouterd-1128330706-25p37
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info retrieved listeners for
> Router.qdrouterd-1128330706-25p37: 172.17.0.10:55672
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info
> Router.qdrouterd-1128330706-25p37 not ready for connectivity check:
> false false
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info EXTRACT, results:
>
> [..]
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info retrieved addresses for
> Router.qdrouterd-1128330706-25p37:
>
> [..]
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info retrieved link routes for
> Router.qdrouterd-1128330706-25p37: raw-> [..]
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info retrieved link routes for
> Router.qdrouterd-1128330706-25p37:
> {"event/DEFAULT_TENANT_in":{"name":"event/
>
> [..]
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info updating addresses for
> Router.qdrouterd-1128330706-25p37
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info
> Router.qdrouterd-1128330706-25p37 not ready for connectivity check: true
> false
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info retrieved connectors for
> Router.qdrouterd-1128330706-25p37:
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info checking connectivity for
> Router.qdrouterd-1128330706-25p37
>
> Mon, 07 Aug 2017 14:55:47 GMT ragent info checking connectors on router
> Router.qdrouterd-1128330706-25p37, missing=, stale=
>
> ----
>
> queue-scheduler:
>
> ----
>
> 2017-08-07 14:53:39 WARN MacAddressUtil:136 - Failed to find a usable
> hardware address from the network interfaces; using random bytes:
> e0:37:7a:1d:7d:a2:98:48
>
> 2017-08-07 14:53:40 INFO QueueScheduler:95 - QueueScheduler is up and
> running
>
> 2017-08-07 14:53:40 ERROR ConfigServiceClient:65 - Error connecting to
> configuration service
>
> io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
> refused: localhost/127.0.0.1:5672
>
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
>
> at
> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:347)
>
> at
> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340)
>
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:630)
>
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:565)
>
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:479)
>
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:441)
>
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
>
> at java.lang.Thread.run(Thread.java:748)
>
> 2017-08-07 14:53:50 INFO ConfigServiceClient:45 - Connected to the
> configuration service
>
> ----
>
> Any idea on what the reason could be or how to investigate further?
>
> Best regards
>
> *Carsten Lohmann
> *
> (INST/ECS4)
> Bosch Software Innovations GmbH | Schöneberger Ufer 89-91 | 10785 Berlin
> | GERMANY| www.bosch-si.com <http://www.bosch-si.com>
> Tel. +49 30 726112-130 | Fax +49 30 726112-100 |
> carsten.lohmann at bosch-si.com <mailto:carsten.lohmann at bosch-si.com>
>
> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung:
> Dr.-Ing. Rainer Kallenbach, Michael Hahn
>
>
>
>
>
> _______________________________________________
> enmasse mailing list
> enmasse at redhat.com
> https://www.redhat.com/mailman/listinfo/enmasse
>
--
Ulf
More information about the enmasse
mailing list