[Strimzi] Strimzi with Prometheus
Paolo Patierno
ppatiern at redhat.com
Tue Aug 6 15:36:18 UTC 2019
Are you able to access the prometheus server web UI?
Can you provide the prometheus server log?
On Tue, Aug 6, 2019 at 5:28 PM Das, Debdeep <debdeep.das at in.unisys.com>
wrote:
> D:\DD\Office\Delivery\Projects\Falcon\kafka\strimzi-kafka-operator\metrics\examples\prometheus\additional-properties>kubectl
> create secret generic additional-scrape-configs
> --from-file=prometheus-additional.yaml -n cattle-prometheus
>
> secret/additional-scrape-configs created
>
>
>
> D:\DD\Office\Delivery\Projects\Falcon\kafka\strimzi-kafka-operator\metrics\examples\prometheus\additional-properties>kubectl
> get secret -n cattle-prometheus
>
> NAME
> TYPE
> DATA AGE
>
> additional-scrape-configs
> Opaque 1 21m
>
>
>
>
>
>
>
> D:\DD\Office\Delivery\Projects\Falcon\kafka\strimzi-kafka-operator\metrics\examples\prometheus\install>kubectl
> apply -f strimzi-service-monitor.yaml -n cattle-prometheus
>
> servicemonitor.monitoring.coreos.com/kafka-service-monitor created
>
>
>
> D:\DD\Office\Delivery\Projects\Falcon\kafka\strimzi-kafka-operator\metrics\examples\prometheus\install>kubectl
> apply -f prometheus-rules.yaml -n cattle-prometheus
>
> prometheusrule.monitoring.coreos.com/prometheus-k8s-rules created
>
>
>
> D:\DD\Office\Delivery\Projects\Falcon\kafka\strimzi-kafka-operator\metrics\examples\prometheus\install>kubectl
> apply -f prometheus.yaml -n cattle-prometheus
>
> clusterrole.rbac.authorization.k8s.io/prometheus-server created
>
> serviceaccount/prometheus-server created
>
> clusterrolebinding.rbac.authorization.k8s.io/prometheus-server created
>
> prometheus.monitoring.coreos.com/prometheus created
>
>
>
>
>
> I still don’t see any metrics for Kafka.
>
>
>
>
>
> *Debdeep Das* | Lead Engineer | Communication Products - InTC
>
> Unisys | +91 9980101186 | debdeep.das at in.unisys.com
>
>
>
> [image: unisys_logo] <http://www.unisys.com/>
>
>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
> MATERIAL and is for use only by the intended recipient. If you received
> this in error, please contact the sender and delete the e-mail and its
> attachments from all devices.
>
> [image: Grey_LI] <http://www.linkedin.com/company/unisys> [image:
> Grey_TW] <http://twitter.com/unisyscorp> [image: Grey_YT]
> <http://www.youtube.com/theunisyschannel>[image: Grey_FB]
> <http://www.facebook.com/unisyscorp>[image: Grey_Vimeo]
> <https://vimeo.com/unisys>[image: Grey_UB] <http://blogs.unisys.com/>
>
>
>
> *From:* Stanislav Knot [mailto:sknot at redhat.com]
> *Sent:* Tuesday, August 6, 2019 8:23 PM
> *To:* Das, Debdeep <debdeep.das at in.unisys.com>
> *Cc:* Paolo Patierno <ppatiern at redhat.com>; strimzi at redhat.com
> *Subject:* Re: [Strimzi] Strimzi with Prometheus
>
>
>
> You can use
>
> kubectl create secret generic additional-scrape-configs
> --from-file=prometheus-additional.yaml -n your-namespace
>
>
>
> Stanislav
>
>
>
> On Tue, Aug 6, 2019 at 4:20 PM Das, Debdeep <debdeep.das at in.unisys.com>
> wrote:
>
> I am using Strimzi 0.12.2
>
>
>
> I created the secret
>
>
>
> kubectl create secret generic additional-scrape-configs
> --from-file=prometheus-additional.yaml
>
>
>
> The next step is to get the secret name. How to do that?
>
>
>
>
>
> *Debdeep Das* | Lead Engineer | Communication Products - InTC
>
> Unisys | +91 9980101186 | debdeep.das at in.unisys.com
>
>
>
> [image: unisys_logo] <http://www.unisys.com/>
>
>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
> MATERIAL and is for use only by the intended recipient. If you received
> this in error, please contact the sender and delete the e-mail and its
> attachments from all devices.
>
> [image: Grey_LI] <http://www.linkedin.com/company/unisys> [image:
> Grey_TW] <http://twitter.com/unisyscorp> [image: Grey_YT]
> <http://www.youtube.com/theunisyschannel>[image: Grey_FB]
> <http://www.facebook.com/unisyscorp>[image: Grey_Vimeo]
> <https://vimeo.com/unisys>[image: Grey_UB] <http://blogs.unisys.com/>
>
>
>
> *From:* Paolo Patierno [mailto:ppatiern at redhat.com]
> *Sent:* Tuesday, August 6, 2019 7:42 PM
> *To:* Das, Debdeep <debdeep.das at in.unisys.com>
> *Cc:* strimzi at redhat.com
> *Subject:* Re: [Strimzi] Strimzi with Prometheus
>
>
>
> As you can see from the log as I thought you are missing the additional
> scrape configs creation.
>
>
>
> On Tue, Aug 6, 2019 at 3:43 PM Das, Debdeep <debdeep.das at in.unisys.com>
> wrote:
>
>
>
> Please find attached the Prometheus operator log.
>
>
>
> *Debdeep Das* | Lead Engineer | Communication Products - InTC
>
> Unisys | +91 9980101186 | debdeep.das at in.unisys.com
>
>
>
> [image: unisys_logo] <http://www.unisys.com/>
>
>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
> MATERIAL and is for use only by the intended recipient. If you received
> this in error, please contact the sender and delete the e-mail and its
> attachments from all devices.
>
> [image: Grey_LI] <http://www.linkedin.com/company/unisys> [image:
> Grey_TW] <http://twitter.com/unisyscorp> [image: Grey_YT]
> <http://www.youtube.com/theunisyschannel>[image: Grey_FB]
> <http://www.facebook.com/unisyscorp>[image: Grey_Vimeo]
> <https://vimeo.com/unisys>[image: Grey_UB] <http://blogs.unisys.com/>
>
>
>
> *From:* Paolo Patierno [mailto:ppatiern at redhat.com]
> *Sent:* Monday, August 5, 2019 8:57 PM
> *To:* Das, Debdeep <debdeep.das at in.unisys.com>
> *Cc:* strimzi at redhat.com
> *Subject:* Re: [Strimzi] Strimzi with Prometheus
>
>
>
> Can you provide me the prometheus-operator log?
>
> From the above steps you are missing the creation of the Secret containing
> the additional configs and maybe the Prometheus operator is failing to
> search for it.
>
> This Secret is referenced in the prometheus.yaml` file as following:
>
>
>
> additionalScrapeConfigs:
>
> name: additional-scrape-configs
>
> key: prometheus-additional.yaml
>
>
>
> The Secret creation is described here:
> https://strimzi.io/docs/latest/#proc-metrics-deploying-prometheus-str
> (it's from master doc, what's the Strimzi version you are using?)
>
>
>
> After that, access to the prometheus web UI and let me know if you can see
> metrics starting with kafka_ and zookeeper_
>
>
>
>
>
> On Mon, Aug 5, 2019 at 12:30 PM Das, Debdeep <debdeep.das at in.unisys.com>
> wrote:
>
> The prometheus server is also running in the cattle-Prometheus namespace.
>
>
>
> *Debdeep Das* | Lead Engineer | Communication Products - InTC
>
> Unisys | +91 9980101186 | debdeep.das at in.unisys.com
>
>
>
> [image: unisys_logo] <http://www.unisys.com/>
>
>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
> MATERIAL and is for use only by the intended recipient. If you received
> this in error, please contact the sender and delete the e-mail and its
> attachments from all devices.
>
> [image: Grey_LI] <http://www.linkedin.com/company/unisys> [image:
> Grey_TW] <http://twitter.com/unisyscorp> [image: Grey_YT]
> <http://www.youtube.com/theunisyschannel>[image: Grey_FB]
> <http://www.facebook.com/unisyscorp>[image: Grey_Vimeo]
> <https://vimeo.com/unisys>[image: Grey_UB] <http://blogs.unisys.com/>
>
>
>
> *From:* Paolo Patierno [mailto:ppatiern at redhat.com]
> *Sent:* Monday, August 5, 2019 3:47 PM
> *To:* Das, Debdeep <debdeep.das at in.unisys.com>
> *Cc:* strimzi at redhat.com
> *Subject:* Re: [Strimzi] Strimzi with Prometheus
>
>
>
> The prometheus operator is not the prometheus server as I mentioned in the
> previous email.
>
> Did you use the prometheus operator to deploy the prometheus server then?
> Where the prometheus server is running? not the operator ;)
>
>
>
> On Mon, Aug 5, 2019 at 12:14 PM Das, Debdeep <debdeep.das at in.unisys.com>
> wrote:
>
> Hello Paolo,
>
> My apologies for the confusion. I am using Prometheus operator, which is
> already installed at the cluster under the name space cattle-prometheus. My
> assumption is I do not need to install the Prometheus operator again and I
> can push the metrics from the kafka cluster ( deployed in
> strimzi-kafka-operator namespace) to the prometheus operator running in
> cattle-prometheus namespace. Is my assumption correct? If yes, what is the
> steps that I need to follow.
>
>
>
> Regards
>
> Debdeep
>
>
>
> *Debdeep Das* | Lead Engineer | Communication Products - InTC
>
> Unisys | +91 9980101186 | debdeep.das at in.unisys.com
>
>
>
> [image: unisys_logo] <http://www.unisys.com/>
>
>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
> MATERIAL and is for use only by the intended recipient. If you received
> this in error, please contact the sender and delete the e-mail and its
> attachments from all devices.
>
> [image: Grey_LI] <http://www.linkedin.com/company/unisys> [image:
> Grey_TW] <http://twitter.com/unisyscorp> [image: Grey_YT]
> <http://www.youtube.com/theunisyschannel>[image: Grey_FB]
> <http://www.facebook.com/unisyscorp>[image: Grey_Vimeo]
> <https://vimeo.com/unisys>[image: Grey_UB] <http://blogs.unisys.com/>
>
>
>
> *From:* Paolo Patierno [mailto:ppatiern at redhat.com]
> *Sent:* Monday, August 5, 2019 3:39 PM
> *To:* Das, Debdeep <debdeep.das at in.unisys.com>
> *Cc:* strimzi at redhat.com
> *Subject:* Re: [Strimzi] Strimzi with Prometheus
>
>
>
> What's not clear to me now is ... if you are using the Prometheus Operator
> or not.
>
> In the first email you mentioned the Prometheus operator while in the last
> one just Prometheus (the server) which are two different things.
>
> The first one is a Kubernetes operator for deploying Prometheus server and
> services to identify metrics to scrape from other applications; the second
> one is ... the actual Prometheus server.
>
> The steps we provide in the documentation are about having Prometheus
> operator running and deploying a few resources for deploying a Prometheus
> server and the services for monitoring Kafka metrics (all these resources
> are handled by the prometheus operator which than "inject" the
> configuration into prometheus).
>
>
>
> On Mon, Aug 5, 2019 at 12:01 PM Das, Debdeep <debdeep.das at in.unisys.com>
> wrote:
>
> Hello Paolo,
>
> Let me explain the environment I have and maybe there is something not
> setup right. I am using Rancher on Kubernetes. On the environment
> Prometheus and Grafana runs on a separate namespace “cattle-prometheus”. My
> Kafka cluster is deployed in strimzi-kafka-operator namespace. I want to
> produce the metrics of the kafka cluster in strimzi-kafka-operator name
> space to the prometheus instance in cattle-prometheus namespace. I am not
> sure what are the steps to do so. If you can help me with the steps to do
> so, it will be great. I could not figure that from the documentation.
>
>
>
> Regards
>
> Debdeep
>
>
>
> *Debdeep Das* | Lead Engineer | Communication Products - InTC
>
> Unisys | +91 9980101186 | debdeep.das at in.unisys.com
>
>
>
> [image: unisys_logo] <http://www.unisys.com/>
>
>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
> MATERIAL and is for use only by the intended recipient. If you received
> this in error, please contact the sender and delete the e-mail and its
> attachments from all devices.
>
> [image: Grey_LI] <http://www.linkedin.com/company/unisys> [image:
> Grey_TW] <http://twitter.com/unisyscorp> [image: Grey_YT]
> <http://www.youtube.com/theunisyschannel>[image: Grey_FB]
> <http://www.facebook.com/unisyscorp>[image: Grey_Vimeo]
> <https://vimeo.com/unisys>[image: Grey_UB] <http://blogs.unisys.com/>
>
>
>
> *From:* Paolo Patierno [mailto:ppatiern at redhat.com]
> *Sent:* Monday, August 5, 2019 1:33 PM
> *To:* Das, Debdeep <debdeep.das at in.unisys.com>
> *Cc:* strimzi at redhat.com
> *Subject:* Re: [Strimzi] Strimzi with Prometheus
>
>
>
> Hi Debdeep,
>
> can you provide the Prometheus log?
>
> In the prometheus.yaml file, the ClusterRoleBinding resource provides the
> binding between the promethes-server service account and the cluster role
> but specifing "myproject" as namespace. Is you Kafka cluster running in the
> myproject namespace? Is the same for the Prometheus server you installed
> through the Prometheus operator?
>
>
>
> Thanks,
>
> Paolo.
>
>
>
> On Mon, Jul 29, 2019 at 5:36 PM Das, Debdeep <debdeep.das at in.unisys.com>
> wrote:
>
> Hello,
>
> I have a Kubernetes environment with Strimzi and Kafka deployed. I am able
> to produce and consume successfully. I want to use Prometheus and Grafana
> to report metrics. The environment already has Prometheus Operator and
> Grafana installed, but in a different namespace than where I installed the
> Strimzi + Kafka.
>
> I used the instructions here https://strimzi.io/docs/latest/#prometheus
> to add the rules. I used the following instructions under the namespace
> where Strimzi-Kafka is installed, but I don’t see the Rules under
> Prometheus. The metrics are already added in the YAML to deploy the
> Kafka(+zookeper)
>
>
>
> kubectl apply -f strimzi-service-monitor.yaml
>
> kubectl apply -f prometheus-rules.yaml
>
> kubectl apply -f prometheus.yaml
>
> Can you please help what needs to be done to get the rules. I guess
> because the rules are not getting to Prometheus, the Grafana is not
> reporting anything on the Dashboard.
>
>
>
> Regards
>
> Debdeep
>
>
>
> *Debdeep Das* | Lead Engineer | Communication Products - InTC
>
> Unisys | +91 9980101186 | debdeep.das at in.unisys.com
>
>
>
> [image: unisys_logo] <http://www.unisys.com/>
>
>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
> MATERIAL and is for use only by the intended recipient. If you received
> this in error, please contact the sender and delete the e-mail and its
> attachments from all devices.
>
> [image: Grey_LI] <http://www.linkedin.com/company/unisys> [image:
> Grey_TW] <http://twitter.com/unisyscorp> [image: Grey_YT]
> <http://www.youtube.com/theunisyschannel>[image: Grey_FB]
> <http://www.facebook.com/unisyscorp>[image: Grey_Vimeo]
> <https://vimeo.com/unisys>[image: Grey_UB] <http://blogs.unisys.com/>
>
>
>
> _______________________________________________
> Strimzi mailing list
> Strimzi at redhat.com
> https://www.redhat.com/mailman/listinfo/strimzi
>
>
>
>
> --
>
> *Paolo Patierno*
>
> Principal Software Engineer, Messaging & IoT
>
> Red Hat <https://www.redhat.com/>
>
> <https://red.ht/sig>
>
> <https://redhat.com/summit>
>
>
>
>
>
>
> --
>
> *Paolo Patierno*
>
> Principal Software Engineer, Messaging & IoT
>
> Red Hat <https://www.redhat.com/>
>
> <https://red.ht/sig>
>
> <https://redhat.com/summit>
>
>
>
>
>
>
> --
>
> *Paolo Patierno*
>
> Principal Software Engineer, Messaging & IoT
>
> Red Hat <https://www.redhat.com/>
>
> <https://red.ht/sig>
>
> <https://redhat.com/summit>
>
>
>
>
>
>
> --
>
> *Paolo Patierno*
>
> Principal Software Engineer, Messaging & IoT
>
> Red Hat <https://www.redhat.com/>
>
> <https://red.ht/sig>
>
> <https://redhat.com/summit>
>
>
>
>
>
>
> --
>
> *Paolo Patierno*
>
> Principal Software Engineer, Messaging & IoT
>
> Red Hat <https://www.redhat.com/>
>
> <https://red.ht/sig>
>
> <https://redhat.com/summit>
>
>
>
> _______________________________________________
> Strimzi mailing list
> Strimzi at redhat.com
> https://www.redhat.com/mailman/listinfo/strimzi
>
>
--
Paolo Patierno
Principal Software Engineer, Messaging & IoT
Red Hat <https://www.redhat.com/>
<https://red.ht/sig>
<https://redhat.com/summit>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/strimzi/attachments/20190806/3833da98/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 4550 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/strimzi/attachments/20190806/3833da98/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 778 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/strimzi/attachments/20190806/3833da98/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 776 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/strimzi/attachments/20190806/3833da98/attachment-0001.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.jpg
Type: image/jpeg
Size: 775 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/strimzi/attachments/20190806/3833da98/attachment-0002.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.jpg
Type: image/jpeg
Size: 755 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/strimzi/attachments/20190806/3833da98/attachment-0003.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.jpg
Type: image/jpeg
Size: 1737 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/strimzi/attachments/20190806/3833da98/attachment-0004.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image007.jpg
Type: image/jpeg
Size: 791 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/strimzi/attachments/20190806/3833da98/attachment-0005.jpg>
More information about the Strimzi
mailing list