[Strimzi] Strimzi with Prometheus

Tue Aug 6 14:21:08 UTC 2019

It has to be created in the namespace where your prometheus operator is
running.
The operator needs that secret for creating the prometheus instance.

On Tue, Aug 6, 2019 at 4:17 PM Stanislav Knot <sknot at redhat.com> wrote:

> I can see in the log you provided `secret additional-scrape-configs could
> not be found`.
> Are you sure you have created the secrets correctly?
>
> Stanislav
>
> On Tue, Aug 6, 2019 at 4:06 PM Das, Debdeep <debdeep.das at in.unisys.com>
> wrote:
>
>>
>>
>> Please find attached the Prometheus operator log.
>>
>>
>>
>> *Debdeep Das* | Lead Engineer | Communication Products - InTC
>>
>> Unisys | +91 9980101186 | debdeep.das at in.unisys.com
>>
>>
>>
>> [image: unisys_logo] <http://www.unisys.com/>
>>
>>
>>
>> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
>> MATERIAL and is for use only by the intended recipient. If you received
>> this in error, please contact the sender and delete the e-mail and its
>> attachments from all devices.
>>
>> [image: Grey_LI] <http://www.linkedin.com/company/unisys>  [image:
>> Grey_TW] <http://twitter.com/unisyscorp> [image: Grey_YT]
>> <http://www.youtube.com/theunisyschannel>[image: Grey_FB]
>> <http://www.facebook.com/unisyscorp>[image: Grey_Vimeo]
>> <https://vimeo.com/unisys>[image: Grey_UB] <http://blogs.unisys.com/>
>>
>>
>>
>> *From:* Paolo Patierno [mailto:ppatiern at redhat.com]
>> *Sent:* Monday, August 5, 2019 8:57 PM
>> *To:* Das, Debdeep <debdeep.das at in.unisys.com>
>> *Cc:* strimzi at redhat.com
>> *Subject:* Re: [Strimzi] Strimzi with Prometheus
>>
>>
>>
>> Can you provide me the prometheus-operator log?
>>
>> From the above steps you are missing the creation of the Secret
>> containing the additional configs and maybe the Prometheus operator is
>> failing to search for it.
>>
>> This Secret is referenced in the prometheus.yaml` file as following:
>>
>>
>>
>> additionalScrapeConfigs:
>>
>>     name: additional-scrape-configs
>>
>>     key: prometheus-additional.yaml
>>
>>
>>
>> The Secret creation is described here:
>> https://strimzi.io/docs/latest/#proc-metrics-deploying-prometheus-str
>> (it's from master doc, what's the Strimzi version you are using?)
>>
>>
>>
>> After that, access to the prometheus web UI and let me know if you can
>> see metrics starting with kafka_ and zookeeper_
>>
>>
>>
>>
>>
>> On Mon, Aug 5, 2019 at 12:30 PM Das, Debdeep <debdeep.das at in.unisys.com>
>> wrote:
>>
>> The prometheus server is also running in the cattle-Prometheus namespace.
>>
>>
>>
>> *Debdeep Das* | Lead Engineer | Communication Products - InTC
>>
>> Unisys | +91 9980101186 | debdeep.das at in.unisys.com
>>
>>
>>
>> [image: unisys_logo] <http://www.unisys.com/>
>>
>>
>>
>> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
>> MATERIAL and is for use only by the intended recipient. If you received
>> this in error, please contact the sender and delete the e-mail and its
>> attachments from all devices.
>>
>> [image: Grey_LI] <http://www.linkedin.com/company/unisys>  [image:
>> Grey_TW] <http://twitter.com/unisyscorp> [image: Grey_YT]
>> <http://www.youtube.com/theunisyschannel>[image: Grey_FB]
>> <http://www.facebook.com/unisyscorp>[image: Grey_Vimeo]
>> <https://vimeo.com/unisys>[image: Grey_UB] <http://blogs.unisys.com/>
>>
>>
>>
>> *From:* Paolo Patierno [mailto:ppatiern at redhat.com]
>> *Sent:* Monday, August 5, 2019 3:47 PM
>> *To:* Das, Debdeep <debdeep.das at in.unisys.com>
>> *Cc:* strimzi at redhat.com
>> *Subject:* Re: [Strimzi] Strimzi with Prometheus
>>
>>
>>
>> The prometheus operator is not the prometheus server as I mentioned in
>> the previous email.
>>
>> Did you use the prometheus operator to deploy the prometheus server then?
>> Where the prometheus server is running? not the operator ;)
>>
>>
>>
>> On Mon, Aug 5, 2019 at 12:14 PM Das, Debdeep <debdeep.das at in.unisys.com>
>> wrote:
>>
>> Hello Paolo,
>>
>> My apologies for the confusion. I am using Prometheus operator, which is
>> already installed at the cluster under the name space cattle-prometheus. My
>> assumption is I do not need to install the Prometheus operator again and I
>> can push the metrics from the kafka cluster ( deployed in
>> strimzi-kafka-operator namespace) to the prometheus operator running in
>> cattle-prometheus namespace. Is my assumption correct? If yes, what is the
>> steps that I need to follow.
>>
>>
>>
>> Regards
>>
>> Debdeep
>>
>>
>>
>> *Debdeep Das* | Lead Engineer | Communication Products - InTC
>>
>> Unisys | +91 9980101186 | debdeep.das at in.unisys.com
>>
>>
>>
>> [image: unisys_logo] <http://www.unisys.com/>
>>
>>
>>
>> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
>> MATERIAL and is for use only by the intended recipient. If you received
>> this in error, please contact the sender and delete the e-mail and its
>> attachments from all devices.
>>
>> [image: Grey_LI] <http://www.linkedin.com/company/unisys>  [image:
>> Grey_TW] <http://twitter.com/unisyscorp> [image: Grey_YT]
>> <http://www.youtube.com/theunisyschannel>[image: Grey_FB]
>> <http://www.facebook.com/unisyscorp>[image: Grey_Vimeo]
>> <https://vimeo.com/unisys>[image: Grey_UB] <http://blogs.unisys.com/>
>>
>>
>>
>> *From:* Paolo Patierno [mailto:ppatiern at redhat.com]
>> *Sent:* Monday, August 5, 2019 3:39 PM
>> *To:* Das, Debdeep <debdeep.das at in.unisys.com>
>> *Cc:* strimzi at redhat.com
>> *Subject:* Re: [Strimzi] Strimzi with Prometheus
>>
>>
>>
>> What's not clear to me now is ... if you are using the Prometheus
>> Operator or not.
>>
>> In the first email you mentioned the Prometheus operator while in the
>> last one just Prometheus (the server) which are two different things.
>>
>> The first one is a Kubernetes operator for deploying Prometheus server
>> and services to identify metrics to scrape from other applications; the
>> second one is ... the actual Prometheus server.
>>
>> The steps we provide in the documentation are about having Prometheus
>> operator running and deploying a few resources for deploying a Prometheus
>> server and the services for monitoring Kafka metrics (all these resources
>> are handled by the prometheus operator which than "inject" the
>> configuration into prometheus).
>>
>>
>>
>> On Mon, Aug 5, 2019 at 12:01 PM Das, Debdeep <debdeep.das at in.unisys.com>
>> wrote:
>>
>> Hello Paolo,
>>
>> Let me explain the environment I have and maybe there is something not
>> setup right. I am using Rancher on Kubernetes. On the environment
>> Prometheus and Grafana runs on a separate namespace “cattle-prometheus”. My
>> Kafka cluster is deployed in strimzi-kafka-operator namespace. I want to
>> produce the metrics of the kafka cluster in strimzi-kafka-operator name
>> space to the prometheus instance in cattle-prometheus namespace. I am not
>> sure what are the steps to do so. If you can help me with the steps to do
>> so, it will be great. I could not figure that from the documentation.
>>
>>
>>
>> Regards
>>
>> Debdeep
>>
>>
>>
>> *Debdeep Das* | Lead Engineer | Communication Products - InTC
>>
>> Unisys | +91 9980101186 | debdeep.das at in.unisys.com
>>
>>
>>
>> [image: unisys_logo] <http://www.unisys.com/>
>>
>>
>>
>> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
>> MATERIAL and is for use only by the intended recipient. If you received
>> this in error, please contact the sender and delete the e-mail and its
>> attachments from all devices.
>>
>> [image: Grey_LI] <http://www.linkedin.com/company/unisys>  [image:
>> Grey_TW] <http://twitter.com/unisyscorp> [image: Grey_YT]
>> <http://www.youtube.com/theunisyschannel>[image: Grey_FB]
>> <http://www.facebook.com/unisyscorp>[image: Grey_Vimeo]
>> <https://vimeo.com/unisys>[image: Grey_UB] <http://blogs.unisys.com/>
>>
>>
>>
>> *From:* Paolo Patierno [mailto:ppatiern at redhat.com]
>> *Sent:* Monday, August 5, 2019 1:33 PM
>> *To:* Das, Debdeep <debdeep.das at in.unisys.com>
>> *Cc:* strimzi at redhat.com
>> *Subject:* Re: [Strimzi] Strimzi with Prometheus
>>
>>
>>
>> Hi Debdeep,
>>
>> can you provide the Prometheus log?
>>
>> In the prometheus.yaml file, the ClusterRoleBinding resource provides the
>> binding between the promethes-server service account and the cluster role
>> but specifing "myproject" as namespace. Is you Kafka cluster running in the
>> myproject namespace? Is the same for the Prometheus server you installed
>> through the Prometheus operator?
>>
>>
>>
>> Thanks,
>>
>> Paolo.
>>
>>
>>
>> On Mon, Jul 29, 2019 at 5:36 PM Das, Debdeep <debdeep.das at in.unisys.com>
>> wrote:
>>
>> Hello,
>>
>> I have a Kubernetes environment with Strimzi and Kafka deployed. I am
>> able to produce and consume successfully. I want to use Prometheus and
>> Grafana to report metrics. The environment already has Prometheus Operator
>> and Grafana installed, but in a different namespace than where I installed
>> the Strimzi + Kafka.
>>
>> I used the instructions here https://strimzi.io/docs/latest/#prometheus
>> to add the rules.  I used the following instructions under the namespace
>> where Strimzi-Kafka is installed, but I don’t see the Rules under
>> Prometheus. The metrics are already added in the YAML to deploy the
>> Kafka(+zookeper)
>>
>>
>>
>> kubectl apply -f strimzi-service-monitor.yaml
>>
>> kubectl apply -f prometheus-rules.yaml
>>
>> kubectl apply -f prometheus.yaml
>>
>>  Can you please help what needs to be done to get the rules. I guess
>> because the rules are not getting to Prometheus, the Grafana is not
>> reporting anything on the Dashboard.
>>
>>
>>
>> Regards
>>
>> Debdeep
>>
>>
>>
>> *Debdeep Das* | Lead Engineer | Communication Products - InTC
>>
>> Unisys | +91 9980101186 | debdeep.das at in.unisys.com
>>
>>
>>
>> [image: unisys_logo] <http://www.unisys.com/>
>>
>>
>>
>> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
>> MATERIAL and is for use only by the intended recipient. If you received
>> this in error, please contact the sender and delete the e-mail and its
>> attachments from all devices.
>>
>> [image: Grey_LI] <http://www.linkedin.com/company/unisys>  [image:
>> Grey_TW] <http://twitter.com/unisyscorp> [image: Grey_YT]
>> <http://www.youtube.com/theunisyschannel>[image: Grey_FB]
>> <http://www.facebook.com/unisyscorp>[image: Grey_Vimeo]
>> <https://vimeo.com/unisys>[image: Grey_UB] <http://blogs.unisys.com/>
>>
>>
>>
>> _______________________________________________
>> Strimzi mailing list
>> Strimzi at redhat.com
>> https://www.redhat.com/mailman/listinfo/strimzi
>>
>>
>>
>>
>> --
>>
>> *Paolo Patierno*
>>
>> Principal Software Engineer, Messaging & IoT
>>
>> Red Hat <https://www.redhat.com/>
>>
>> <https://red.ht/sig>
>>
>> <https://redhat.com/summit>
>>
>>
>>
>>
>>
>>
>> --
>>
>> *Paolo Patierno*
>>
>> Principal Software Engineer, Messaging & IoT
>>
>> Red Hat <https://www.redhat.com/>
>>
>> <https://red.ht/sig>
>>
>> <https://redhat.com/summit>
>>
>>
>>
>>
>>
>>
>> --
>>
>> *Paolo Patierno*
>>
>> Principal Software Engineer, Messaging & IoT
>>
>> Red Hat <https://www.redhat.com/>
>>
>> <https://red.ht/sig>
>>
>> <https://redhat.com/summit>
>>
>>
>>
>>
>>
>>
>> --
>>
>> *Paolo Patierno*
>>
>> Principal Software Engineer, Messaging & IoT
>>
>> Red Hat <https://www.redhat.com/>
>>
>> <https://red.ht/sig>
>>
>> <https://redhat.com/summit>
>>
>>
>> _______________________________________________
>> Strimzi mailing list
>> Strimzi at redhat.com
>> https://www.redhat.com/mailman/listinfo/strimzi
>>
>

-- 

Paolo Patierno

Principal Software Engineer, Messaging & IoT

Red Hat <https://www.redhat.com/>
<https://red.ht/sig>
<https://redhat.com/summit>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/strimzi/attachments/20190806/58decf56/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 4550 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/strimzi/attachments/20190806/58decf56/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 778 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/strimzi/attachments/20190806/58decf56/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 776 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/strimzi/attachments/20190806/58decf56/attachment-0001.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.jpg
Type: image/jpeg
Size: 775 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/strimzi/attachments/20190806/58decf56/attachment-0002.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.jpg
Type: image/jpeg
Size: 755 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/strimzi/attachments/20190806/58decf56/attachment-0003.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.jpg
Type: image/jpeg
Size: 1737 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/strimzi/attachments/20190806/58decf56/attachment-0004.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image007.jpg
Type: image/jpeg
Size: 791 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/strimzi/attachments/20190806/58decf56/attachment-0005.jpg>