[Tendrl-devel] installation of monitoring and alerting

[Jeff Applewhite]

For now why don't we keep the installation documentation and pathway
simple.

 This would mean a a single Tendrl node including all required components
with a *manual* (no Ansible) process to setup HA if the admin so desires.
This would involved replicating etcd to two (or more) other nodes primarily
(since the failover process has also not been tested yet). This is simply
so that in the case of primary node failure a recovery could be achieved
based on a replica etcd node. No need to setup API on the secondary at this
point. That can come later as we prove out those scenarios.

Let's also put a *note* in the documentation that running performance
monitoring on another node is theoretically possible but not tested.

On Wed, May 31, 2017 at 6:20 AM, Anmol Babu <anbabu at redhat.com> wrote:

> Please also note,
> if we decide to put performance-monitoring and graphite on different
> nodes(in case graphite is already deployed and is in use even before tendrl
> installation),
> I would like to say that this can be achieved with no to very less(might
> require some spec changes but needs to be verified thoroughly though)
> changes in performance-monitoring code-base..
>
> Anmol Babu
> Software Engineer
> Red Hat India Pvt Ltd
> M: 8884245644
>
> ----- Original Message -----
> From: "Anmol Babu" <anbabu at redhat.com>
> To: "Mailing list for the contributors to the Tendrl project" <
> tendrl-devel at redhat.com>
> Sent: Wednesday, May 31, 2017 3:37:41 PM
> Subject: Re: [Tendrl-devel] installation of monitoring and alerting
>
> Hi Martin,
>
> As an implementer I have always been trying to keep maximum options open
> in the code and to suit timelines so that any future changes are easy to
> adapt to.
> And doing so, I totally missed this in the midst of deliverable s and
> timelines..
> Thanks a lot for bringing up this discussion here...
>
> Important points to note before diving into the questions:
>
> 1. The performance-monitoring installation brings in(installs) and
> configures graphite.
> 2. All nodes push stats to the graphite installed by
> performance-monitoring application..
> 3. Writes to graphite induce read/write disk operations as the stats are
> maintained in files by whisper database(part of graphite stack).
>    Note: Metrics get fed into the graphite stack via the Carbon service,
> which writes data out to Whisper databases for long-term storage.
>
> Now the questions:
>
> >1) What are pros and cons of each installation configuration? Based on
> >   what should one decide on which machine to install the role in both
> >   cases?
>
> I think the answer to this is mainly governed by:
> a. Points 1 to 3 above under "Important points to note..."
> b. performance-monitoring application does lots of etcd interactions and
> also has interactions from tendrl/api which makes me say that having them
> co-resident
>    would be beneficial from network utilization perspective may not be a
> major gain though...
> c. To avoid having to dedicate a node for this purpose one can as well use
> the same node for installing etcd, api, dashboard, performance-monitoring
> and alerting application along with their dependent service node-agent.
> d. Point c in essence also suggests that this can be essentially deployed
> on a storage node as well.
>    Note: Having etcd and api along with performance-monitoring,
> node-monitoring and alerting might bring in resource utilization related
> issues.
>          But this might need to be confirmed via tests...
>
> >2) What is suggested safe default?
>
> The answer to this varies in accordance with considerations above...
>
> >3) Does "New node" for Performance Monitoring mean a dedicated machine
> >   just for this role?
>
> Yes, based on considerations above, if this is inevitable, there is
> nothing in code that stops such a deployment...
>
> >4) Why alternative place for monitoring is "new node", while alerting
> >   could be placed on storage node? Is it possible to install monitoring
> >   and alerting on single dedicated machine? And would it make sense?
>
> The alternatives to both applications from the perspective of code are the
> same..
> Its just a fix required on documentation if it suggests otherwise..
> Having performance-monitoring and alerting on a dedicated machine might
> make sense but yes as said above it all depends on considerations above...
>
> To summarise, I would like to say that definitive and/or quantitative
> answers to most of these questions, can be made after:
>
> 1. We test each of these deployment scenarios
> 2. We perform scale tests to see how tendrl scales...
>
>
> @Nishanth, @shubhendu, @rohan and @Mrugesh Please provide your valuable
> thoughts...
>
>
> Anmol Babu
> Software Engineer
> Red Hat India Pvt Ltd
> M: 8884245644
>
> ----- Original Message -----
> From: "Martin Bukatovic" <mbukatov at redhat.com>
> To: tendrl-devel at redhat.com
> Sent: Wednesday, May 31, 2017 1:55:05 PM
> Subject: [Tendrl-devel] installation of monitoring and alerting
>
> Dear all,
>
> I have another questiong about Tendrl installation. In "Tendrl Package
> Installation Reference" document[1], both *Performance Moritoring* and
> *Alerting* roles states that one can install mentioned components on
> different machines:
>
> > Performance Monitoring can be installed either on:
> > * Node where etcd is installed
> > * New node
> >
> > Alerting can be installed either on:
> >
> > * Node where etcd is installed
> > * Storage node
>
> I would like to get this clarified a bit.
>
> 1) What are pros and cons of each installation configuration? Based on
>    what should one decide on which machine to install the role in both
>    cases?
> 2) What is suggested safe default?
> 3) Does "New node" for Performance Monitoring mean a dedicated machine
>    just for this role?
> 4) Why alternative place for monitoring is "new node", while alerting
>    could be placed on storage node? Is it possible to install monitoring
>    and alerting on single dedicated machine? And would it make sense?
>
> Thank you for checking this.
>
> --
> Martin Bukatovic
> USM QE team
> Red Hat
>
> _______________________________________________
> Tendrl-devel mailing list
> Tendrl-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/tendrl-devel
>
> _______________________________________________
> Tendrl-devel mailing list
> Tendrl-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/tendrl-devel
>



-- 
Jeff Applewhite
Principal Product Manager