<div dir="ltr">Yeah, you have to install all of them. There is currently no way to tell the Cluster Operator to handle only some of the resources.<div> </div><div>Thanks & Regards</div><div>Jakub</div></div> <div class="gmail_quote"><div dir="ltr">On Mon, Jan 7, 2019 at 3:44 PM Schwarz, Markus <<a href="mailto:M.Schwarz@prosoz.de">M.Schwarz@prosoz.de</a>> wrote: </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div lang="DE"> <div class="gmail-m_-671541413616771147WordSection1"> Hi Jakub, that fixed it indeed. The crds for kafkaconnect + s2i and kafkamirrormakers where missing, I guess I omitted them because I don’t want to use either kafka connect nor mirrormaker. Thanks! Markus Von: Jakub Scholz [mailto:<a href="mailto:jakub@scholz.cz" target="_blank">jakub@scholz.cz</a>] Gesendet: Montag, 7. Januar 2019 15:39 An: Schwarz, Markus <<a href="mailto:M.Schwarz@prosoz.de" target="_blank">M.Schwarz@prosoz.de</a>> Cc: <a href="mailto:strimzi@redhat.com" target="_blank">strimzi@redhat.com</a> Betreff: Re: [Strimzi] Cluster-Operator 0.9.0 does not start due to error 404 on websocket connection <div> <div> Thanks. I did some tests and it seems I get the same error when the Custom Resource Definitions are not properly installed. Can you do "kubectl get crd" and make sure you get something like this: <div> </div> <div> <div> $ kubectl get crd </div> <div> NAME AGE </div> <div> <a href="http://kafkaconnects.kafka.strimzi.io" target="_blank">kafkaconnects.kafka.strimzi.io</a> 5m39s </div> <div> <a href="http://kafkaconnects2is.kafka.strimzi.io" target="_blank">kafkaconnects2is.kafka.strimzi.io</a> 5m39s </div> <div> <a href="http://kafkamirrormakers.kafka.strimzi.io" target="_blank">kafkamirrormakers.kafka.strimzi.io</a> 5m39s </div> <div> <a href="http://kafkas.kafka.strimzi.io" target="_blank">kafkas.kafka.strimzi.io</a> 5m39s </div> <div> <a href="http://kafkatopics.kafka.strimzi.io" target="_blank">kafkatopics.kafka.strimzi.io</a> 5m39s </div> <div> <a href="http://kafkausers.kafka.strimzi.io" target="_blank">kafkausers.kafka.strimzi.io</a> 5m39s </div> <div> </div> <div> Thanks & Regards </div> <div> Jakub </div> <div> </div> </div> </div> </div> <div> <div> On Mon, Jan 7, 2019 at 3:15 PM Schwarz, Markus <<a href="mailto:M.Schwarz@prosoz.de" target="_blank">M.Schwarz@prosoz.de</a>> wrote: </div> <blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm"> <div> <div> Hi Jakub, here is the complete log from the co: + JAR=/cluster-operator-0.9.0.jar + shift + . /bin/dynamic_resources.sh ++ get_heap_size +++ cat /sys/fs/cgroup/memory/memory.limit_in_bytes ++ CONTAINER_MEMORY_IN_BYTES=268435456 ++ DEFAULT_MEMORY_CEILING=1152921504606846975 ++ '[' 268435456 -lt 1152921504606846975 ']' ++ '[' -z ']' ++ CONTAINER_HEAP_PERCENT=0.50 ++ CONTAINER_MEMORY_IN_MB=256 +++ echo '256 0.50' +++ awk '{ printf "%d", $1 * $2 }' ++ CONTAINER_HEAP_MAX=128 ++ echo 128 + MAX_HEAP=128 + '[' -n 128 ']' + JAVA_OPTS='-Xms128m -Xmx128m ' + export MALLOC_ARENA_MAX=2 + MALLOC_ARENA_MAX=2 + JAVA_OPTS='-Xms128m -Xmx128m -Dvertx.cacheDirBase=/tmp -Djava.security.egd=file:/dev/./urandom' + JAVA_OPTS='-Xms128m -Xmx128m -Dvertx.cacheDirBase=/tmp -Djava.security.egd=file:/dev/./urandom -XX:NativeMemoryTracking=summary -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps' + exec java -Xms128m -Xmx128m -Dvertx.cacheDirBase=/tmp -Djava.security.egd=file:/dev/./urandom -XX:NativeMemoryTracking=summary -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -jar /cluster-operator-0.9.0.jar -Xms128m -Xmx128m -Dvertx.cacheDirBase=/tmp -Djava.security.egd=file:/dev/./urandom -XX:NativeMemoryTracking=summary -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps 2019-01-07 14:02:15 INFO Main:70 - ClusterOperator 0.9.0 is starting 2019-01-07T14:02:16.434+0000: [GC (Allocation Failure) 2019-01-07T14:02:16.434+0000: [DefNew: 34944K->4352K(39296K), 0.0154766 secs] 34944K->8729K(126720K), 0.0155884 secs] [Times: user=0.01 sys=0.01, real=0.01 secs] 2019-01-07T14:02:17.040+0000: [Full GC (Metadata GC Threshold) 2019-01-07T14:02:17.040+0000: [Tenured: 4377K->7631K(87424K), 0.0301695 secs] 24907K->7631K(126720K), [Metaspace: 20706K->20706K(1069056K)], 0.0303281 secs] [Times: user=0.03 sys=0.00, real=0.03 secs] 2019-01-07 14:02:17 INFO Main:262 - Using config: PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin KUBERNETES_PORT_53_UDP_PROTO: udp STRIMZI_DEFAULT_KAFKA_MIRRORMAKER_IMAGE: strimzi/kafka-mirror-maker:0.9.0 PROMETHEUS_PORT_9090_TCP: tcp://<a href="http://172.30.38.237:9090" target="_blank">172.30.38.237:9090</a> STRIMZI_FULL_RECONCILIATION_INTERVAL_MS: 120000 STRIMZI_DEFAULT_KAFKA_CONNECT_IMAGE: strimzi/kafka-connect:0.9.0 KUBERNETES_PORT_443_TCP_PROTO: tcp KUBERNETES_PORT_53_TCP_PORT: 53 PROMETHEUS_PORT_9090_TCP_ADDR: 172.30.38.237 KAFKA_METRICS_PORT_9404_TCP_PROTO: tcp STRIMZI_VERSION: 0.9.0 PROMETHEUS_PORT_9090_TCP_PROTO: tcp KUBERNETES_PORT_53_TCP_PROTO: tcp KUBERNETES_PORT_53_TCP_ADDR: 172.30.0.1 HOSTNAME: strimzi-cluster-operator-5d5445c55d-9lb8v HOME: / MALLOC_ARENA_MAX: 2 STRIMZI_DEFAULT_ZOOKEEPER_IMAGE: strimzi/zookeeper:0.9.0 STRIMZI_NAMESPACE: msw KUBERNETES_SERVICE_PORT_HTTPS: 443 SHLVL: 1 JAVA_HOME: /usr/lib/jvm/java KAFKA_METRICS_PORT_9404_TCP_ADDR: 172.30.193.60 STRIMZI_DEFAULT_KAFKA_INIT_IMAGE: strimzi/kafka-init:0.9.0 KAFKA_METRICS_SERVICE_HOST: 172.30.193.60 KUBERNETES_PORT_443_TCP: tcp://<a href="http://172.30.0.1:443" target="_blank">172.30.0.1:443</a> PROMETHEUS_PORT_9090_TCP_PORT: 9090 STRIMZI_OPERATION_TIMEOUT_MS: 300000 PROMETHEUS_SERVICE_PORT: 9090 KAFKA_METRICS_SERVICE_PORT_METRICS: 9404 STRIMZI_DEFAULT_KAFKA_IMAGE: strimzi/kafka:0.9.0 STRIMZI_DEFAULT_TLS_SIDECAR_KAFKA_IMAGE: strimzi/kafka-stunnel:0.9.0 PROMETHEUS_PORT: tcp://<a href="http://172.30.38.237:9090" target="_blank">172.30.38.237:9090</a> STRIMZI_LOG_LEVEL: INFO KUBERNETES_PORT: tcp://<a href="http://172.30.0.1:443" target="_blank">172.30.0.1:443</a> PROMETHEUS_SERVICE_HOST: 172.30.38.237 STRIMZI_DEFAULT_KAFKA_CONNECT_S2I_IMAGE: strimzi/kafka-connect-s2i:0.9.0 KAFKA_METRICS_PORT: tcp://<a href="http://172.30.193.60:9404" target="_blank">172.30.193.60:9404</a> KUBERNETES_PORT_53_TCP: tcp://<a href="http://172.30.0.1:53" target="_blank">172.30.0.1:53</a> KUBERNETES_PORT_53_UDP: udp://<a href="http://172.30.0.1:53" target="_blank">172.30.0.1:53</a> KUBERNETES_SERVICE_PORT: 443 KAFKA_METRICS_SERVICE_PORT: 9404 KUBERNETES_PORT_53_UDP_ADDR: 172.30.0.1 STRIMZI_DEFAULT_TOPIC_OPERATOR_IMAGE: strimzi/topic-operator:0.9.0 PWD: / PROMETHEUS_SERVICE_PORT_PROMETHEUS: 9090 KUBERNETES_PORT_443_TCP_ADDR: 172.30.0.1 STRIMZI_DEFAULT_USER_OPERATOR_IMAGE: strimzi/user-operator:0.9.0 KUBERNETES_SERVICE_PORT_DNS_TCP: 53 STRIMZI_DEFAULT_TLS_SIDECAR_ENTITY_OPERATOR_IMAGE: strimzi/entity-operator-stunnel:0.9.0 KUBERNETES_PORT_53_UDP_PORT: 53 KAFKA_METRICS_PORT_9404_TCP: tcp://<a href="http://172.30.193.60:9404" target="_blank">172.30.193.60:9404</a> KUBERNETES_SERVICE_HOST: 172.30.0.1 KUBERNETES_SERVICE_PORT_DNS: 53 KUBERNETES_PORT_443_TCP_PORT: 443 STRIMZI_DEFAULT_TLS_SIDECAR_ZOOKEEPER_IMAGE: strimzi/zookeeper-stunnel:0.9.0 2019-01-07 14:02:17 INFO ClusterOperator:58 - Creating ClusterOperator for namespace msw 2019-01-07 14:02:17 INFO ClusterOperator:86 - Starting ClusterOperator for namespace msw 2019-01-07 14:02:17 INFO ClusterOperator:93 - Started operator for Kafka kind 2019-01-07 14:02:17 WARN WatchConnectionManager:185 - Exec Failure: HTTP 404, Status: 404 - 404 page not found java.net.ProtocolException: Expected HTTP 101 response but was '404 Not Found' at okhttp3.internal.ws.RealWebSocket.checkResponse(RealWebSocket.java:219) [cluster-operator-0.9.0.jar:0.9.0] at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:186) [cluster-operator-0.9.0.jar:0.9.0] at okhttp3.RealCall$AsyncCall.execute(RealCall.java:153) [cluster-operator-0.9.0.jar:0.9.0] at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) [cluster-operator-0.9.0.jar:0.9.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_191] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_191] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] 2019-01-07 14:02:17 INFO WatchConnectionManager:379 - Current reconnect backoff is 1000 milliseconds (T0) 2019-01-07 14:02:17 ERROR Main:141 - Cluster Operator verticle in namespace msw failed to start io.fabric8.kubernetes.client.KubernetesClientException: 404 page not found at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$2.onFailure(WatchConnectionManager.java:189) ~[cluster-operator-0.9.0.jar:0.9.0] at okhttp3.internal.ws.RealWebSocket.failWebSocket(RealWebSocket.java:546) ~[cluster-operator-0.9.0.jar:0.9.0] at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:188) ~[cluster-operator-0.9.0.jar:0.9.0] at okhttp3.RealCall$AsyncCall.execute(RealCall.java:153) ~[cluster-operator-0.9.0.jar:0.9.0] at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) ~[cluster-operator-0.9.0.jar:0.9.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_191] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_191] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] Heap def new generation total 39296K, used 16240K [0x00000000f8000000, 0x00000000faaa0000, 0x00000000faaa0000) eden space 34944K, 46% used [0x00000000f8000000, 0x00000000f8fdc0d8, 0x00000000fa220000) from space 4352K, 0% used [0x00000000fa660000, 0x00000000fa660000, 0x00000000faaa0000) to space 4352K, 0% used [0x00000000fa220000, 0x00000000fa220000, 0x00000000fa660000) tenured generation total 87424K, used 7631K [0x00000000faaa0000, 0x0000000100000000, 0x0000000100000000) the space 87424K, 8% used [0x00000000faaa0000, 0x00000000fb213ed8, 0x00000000fb214000, 0x0000000100000000) Metaspace used 23303K, capacity 23692K, committed 24064K, reserved 1071104K class space used 2656K, capacity 2770K, committed 2816K, reserved 1048576K Thanks & Regards Markus Von: Jakub Scholz [mailto:<a href="mailto:jakub@scholz.cz" target="_blank">jakub@scholz.cz</a>] Gesendet: Montag, 7. Januar 2019 15:04 An: Schwarz, Markus <<a href="mailto:M.Schwarz@prosoz.de" target="_blank">M.Schwarz@prosoz.de</a>> Cc: <a href="mailto:strimzi@redhat.com" target="_blank">strimzi@redhat.com</a> Betreff: Re: [Strimzi] Cluster-Operator 0.9.0 does not start due to error 404 on websocket connection <div> Hi Markus, <div> </div> <div> 0.9.0 should work fine with with Kubernetes 1.9 / OpenShift 3.9. Could you share the complete log from the CO? The Kubernetes client normally takes the address of the Kubernetes APi from the Kubernetes environment variables and connects there. Maybe in your case there is something strange / wrong with your cluster configuration. </div> <div> </div> <div> Thanks & Regards </div> <div> Jakub </div> </div> <div> <div> On Mon, Jan 7, 2019 at 2:40 PM Schwarz, Markus <<a href="mailto:M.Schwarz@prosoz.de" target="_blank">M.Schwarz@prosoz.de</a>> wrote: </div> <blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt"> <div> <div> Hi, We are currently running strimzi 0.4.0 ( I know, it’s old) on our Openshift Origin 3.9 cluster and everything is working about fine. I know try to update to 0.9.0 to catch up to things and implement some security. So I took all the yaml-files from the cluster-operator install folder, made the necessary namespace amendments and gave it a try. The strimzi-cluster-operator pod tries to start but then dies with the following error message: --- 2019-01-07 12:49:29 WARN WatchConnectionManager:185 - Exec Failure: HTTP 404, Status: 404 - 404 page not found java.net.ProtocolException: Expected HTTP 101 response but was '404 Not Found' at okhttp3.internal.ws.RealWebSocket.checkResponse(RealWebSocket.java:219) [cluster-operator-0.9.0.jar:0.9.0] at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:186) [cluster-operator-0.9.0.jar:0.9.0] at okhttp3.RealCall$AsyncCall.execute(RealCall.java:153) [cluster-operator-0.9.0.jar:0.9.0] at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) [cluster-operator-0.9.0.jar:0.9.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_191] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_191] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] 2019-01-07 12:49:29 INFO WatchConnectionManager:379 - Current reconnect backoff is 1000 milliseconds (T0) 2019-01-07 12:49:29 ERROR Main:141 - Cluster Operator verticle in namespace msw failed to start io.fabric8.kubernetes.client.KubernetesClientException: 404 page not found at io.fabric8.kubernetes.client.dsl.internal.WatchConnectionManager$2.onFailure(WatchConnectionManager.java:189) ~[cluster-operator-0.9.0.jar:0.9.0] at okhttp3.internal.ws.RealWebSocket.failWebSocket(RealWebSocket.java:546) ~[cluster-operator-0.9.0.jar:0.9.0] at okhttp3.internal.ws.RealWebSocket$2.onResponse(RealWebSocket.java:188) ~[cluster-operator-0.9.0.jar:0.9.0] at okhttp3.RealCall$AsyncCall.execute(RealCall.java:153) ~[cluster-operator-0.9.0.jar:0.9.0] at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32) ~[cluster-operator-0.9.0.jar:0.9.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_191] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_191] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191] --- It seems to try to open a websocket connection on a url which does not exists. I don’t know if this is connected to the outdated version of kubernetes (1.9.1) or if this might be a configuration error of some sort, any hint will be appreciated. I could not find any kubernetes/openshift version requirements for strimzi. Thanks! Markus </div> <a href="http://www.prosoz.de" target="_blank"><img border="0" width="555" height="264" id="gmail-m_-671541413616771147gmail-m_-8818134329066022897_x005f_x0000_i1025"></a> prosoz-herten-footer </div> _______________________________________________ Strimzi mailing list <a href="mailto:Strimzi@redhat.com" target="_blank">Strimzi@redhat.com</a> <a href="https://www.redhat.com/mailman/listinfo/strimzi" target="_blank">https://www.redhat.com/mailman/listinfo/strimzi</a> </blockquote> </div> </div> </div> </blockquote> </div> </div> </div> </blockquote></div>