From japplewh at redhat.com Fri May 5 16:38:33 2017 From: japplewh at redhat.com (Jeff Applewhite) Date: Fri, 5 May 2017 12:38:33 -0400 Subject: [Tendrl-devel] broken links on Tendrl.org Message-ID: Hi All It looks like the installation docs have moved and now tendrl.org is broken The old links were: https://github.com/Tendrl/ceph_bridge/blob/master/doc/source/installation.rst https://github.com/Tendrl/gluster_bridge/blob/master/doc/source/installation.rst Thanks? -- Jeff Applewhite Principal Product Manager From anbabu at redhat.com Mon May 8 07:23:09 2017 From: anbabu at redhat.com (Anmol Babu) Date: Mon, 8 May 2017 03:23:09 -0400 (EDT) Subject: [Tendrl-devel] collectd-ping dependency In-Reply-To: <1407529348.4772842.1494228109936.JavaMail.zimbra@redhat.com> Message-ID: <1646331465.4774095.1494228189578.JavaMail.zimbra@redhat.com> Recently we added a new dependency by name collectd-ping for measuring the ping latencies from every node to the performance-monitoring node. Note: PR that adds this dependency : https://github.com/Tendrl/node-monitoring/pull/22 Currently the centos installation of node-monitoring pulls this dependency from epel. We might need to maintain this in our dependencies repo. The version that I am using in my setup(the one pulled from epel) is : [root]# rpm -qa|grep collectd-ping collectd-ping-5.7.1-2.el7.x86_64 Anmol Babu Software Engineer Red Hat India Pvt Ltd M: 8884245644 From mbukatov at redhat.com Tue May 9 08:01:22 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Tue, 9 May 2017 10:01:22 +0200 Subject: [Tendrl-devel] collectd-ping dependency In-Reply-To: <1646331465.4774095.1494228189578.JavaMail.zimbra@redhat.com> References: <1646331465.4774095.1494228189578.JavaMail.zimbra@redhat.com> Message-ID: <5818d62e-4466-9b40-70f0-d487d2797ef4@redhat.com> On 05/08/2017 09:23 AM, Anmol Babu wrote: > Recently we added a new dependency by name collectd-ping for measuring the ping latencies from every node to the performance-monitoring node. > Note: PR that adds this dependency : https://github.com/Tendrl/node-monitoring/pull/22 > > Currently the centos installation of node-monitoring pulls this dependency from epel. > We might need to maintain this in our dependencies repo. > > The version that I am using in my setup(the one pulled from epel) is : > > [root]# rpm -qa|grep collectd-ping > collectd-ping-5.7.1-2.el7.x86_64 Since we already have epel 7 as official dependency for upstream el7 Tendrl builds, using collectd-ping from epel is fine. Moreover rebuilding a package which is already in other repository is not a good idea, unless we have a really good reason to justify the additional work associated with the maintenance. Downstream is a different story, but this is upstream mailing list. -- Martin Bukatovic USM QE team Red Hat From anbabu at redhat.com Tue May 9 08:50:46 2017 From: anbabu at redhat.com (Anmol Babu) Date: Tue, 9 May 2017 04:50:46 -0400 (EDT) Subject: [Tendrl-devel] collectd-ping dependency In-Reply-To: <5818d62e-4466-9b40-70f0-d487d2797ef4@redhat.com> References: <1646331465.4774095.1494228189578.JavaMail.zimbra@redhat.com> <5818d62e-4466-9b40-70f0-d487d2797ef4@redhat.com> Message-ID: <421673276.5161449.1494319846479.JavaMail.zimbra@redhat.com> @Martin Bukatovic, Thanks, for your response. Accordingly, I have initiated the discussion on the other end(Downstream list)... Anmol Babu Software Engineer Red Hat India Pvt Ltd M: 8884245644 ----- Original Message ----- From: "Martin Bukatovic" To: tendrl-devel at redhat.com Sent: Tuesday, May 9, 2017 1:31:22 PM Subject: Re: [Tendrl-devel] collectd-ping dependency On 05/08/2017 09:23 AM, Anmol Babu wrote: > Recently we added a new dependency by name collectd-ping for measuring the ping latencies from every node to the performance-monitoring node. > Note: PR that adds this dependency : https://github.com/Tendrl/node-monitoring/pull/22 > > Currently the centos installation of node-monitoring pulls this dependency from epel. > We might need to maintain this in our dependencies repo. > > The version that I am using in my setup(the one pulled from epel) is : > > [root]# rpm -qa|grep collectd-ping > collectd-ping-5.7.1-2.el7.x86_64 Since we already have epel 7 as official dependency for upstream el7 Tendrl builds, using collectd-ping from epel is fine. Moreover rebuilding a package which is already in other repository is not a good idea, unless we have a really good reason to justify the additional work associated with the maintenance. Downstream is a different story, but this is upstream mailing list. -- Martin Bukatovic USM QE team Red Hat _______________________________________________ Tendrl-devel mailing list Tendrl-devel at redhat.com https://www.redhat.com/mailman/listinfo/tendrl-devel From japplewh at redhat.com Tue May 9 17:05:28 2017 From: japplewh at redhat.com (Jeff Applewhite) Date: Tue, 9 May 2017 13:05:28 -0400 Subject: [Tendrl-devel] Fwd: GSOC on ceph-mgr:Cluster Status Dashboard In-Reply-To: References: Message-ID: ---------- Forwarded message ---------- From: saumay agrawal Date: Tue, May 9, 2017 at 10:30 AM Subject: Fwd: GSOC on ceph-mgr:Cluster Status Dashboard To: ceph-devel at vger.kernel.org Hi everyone, I am Saumay Agrawal, a freshman at Vellore Institute of Technology, Chennai Campus, India. I am pursuing BTech. in Computer Science and Engineering. I am mainly interested in Web Development, UI/UX Design, and Data Storage and Analytics. I take great pleasure in learning new kinds of technologies. I am very glad that I was selected to work with Ceph community. I hope I would be able to make a worthwhile contribution to Ceph. Please do have a look at my proposal, and let me know if I could make it better in any way. I believe feedback is the only way we can learn more and make better things. https://docs.google.com/document/d/14-a9qYu2tniaZADq1EYm3JtFxzKkga7Z 8nLdHV6jC_U/edit?usp=sharing As of now, I am looking into the dashboard prototype almost ready to be merged. https://github.com/ceph/ceph/pull/14946 Saumay. -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo at vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html From khartsoe at redhat.com Wed May 10 13:19:52 2017 From: khartsoe at redhat.com (Kenneth Hartsoe) Date: Wed, 10 May 2017 09:19:52 -0400 (EDT) Subject: [Tendrl-devel] RHS-C 3.0: ansible install method In-Reply-To: <104ced9e-e1b4-5999-86f8-69cb6eea151d@redhat.com> References: <1526008124.12160812.1492095439690.JavaMail.zimbra@redhat.com> <104ced9e-e1b4-5999-86f8-69cb6eea151d@redhat.com> Message-ID: <1059749524.5605525.1494422392624.JavaMail.zimbra@redhat.com> Hi Martin, Thanks for discussing Console 3.0 ansible requirements and the downstream documentation. Per our conversation, my understanding is that there is no ansible support (user tasks) for the downstream documentation for both the aplha and beta delivery drops, and that ansible support for downstream documentation at GA is still TBD, after feedback received during the beta testing cycle. Hi Jeff...pls let me know if this understanding is not in accordance with your delivery plans/expectations. Thanks. Ken Hartsoe Content Strategist Red Hat Storage Documentation khartsoe at redhat.com; IRC: khartsoe Office: 919 754 4770; Internal: 814 4770 ----- Original Message ----- | Hi Ken, | | On 04/13/2017 04:57 PM, Kenneth Hartsoe wrote: | > Looking for ansible playbooks to glean install method for inclusion in the | > RHS-C 3 doc. Please forward link, etc. Thank you. | | there is no downstream ansible based installation for now. There are 2 | upstream ones: one used by | Jeff and other one use and made by QE. There is agreement that installation | process will be more | clear after end of Beta and there will be some kind of merge from both | repositories to | tendrl-ansible repository with ansible playbooks for Tendrl installation. | | | -- | Best Regards, | Martin Kudlej. | RHSC/USM Senior Quality Assurance Engineer | Red Hat Czech s.r.o. | | Phone: +420 532 294 155 | E-mail:mkudlej at redhat.com | IRC: mkudlej at #brno, #gluster, #storage-qa, #rhs, #rh-ceph, #usm-meeting | @ redhat | #tendrl-devel @ freenode | | _______________________________________________ | Tendrl-devel mailing list | Tendrl-devel at redhat.com | https://www.redhat.com/mailman/listinfo/tendrl-devel | From japplewh at redhat.com Wed May 10 13:36:44 2017 From: japplewh at redhat.com (Jeff Applewhite) Date: Wed, 10 May 2017 09:36:44 -0400 Subject: [Tendrl-devel] RHS-C 3.0: ansible install method In-Reply-To: <1059749524.5605525.1494422392624.JavaMail.zimbra@redhat.com> References: <1526008124.12160812.1492095439690.JavaMail.zimbra@redhat.com> <104ced9e-e1b4-5999-86f8-69cb6eea151d@redhat.com> <1059749524.5605525.1494422392624.JavaMail.zimbra@redhat.com> Message-ID: Hi Kenneth The plan QE and Development agreed on was that that the usmqe-setup repo which contains some tendrl-* playbooks would be the source for installation of Tendrl itself and the storage node setup and configuration. The plan was for the upstream build to stabilize (which we are very close to) and then merge the necessary playbooks into a new tendrl-ansible branch (joint dev/qe effort). So if you want to get a sense of what the process will look like the usmqe-setup repo is the place to start now. Once we reach the beta phase development will take over the ownership of the tendrl-ansible repo and those playbooks. One item yet to be defined is how do we do scale out deployments where etcd and the tendrl API is replicated onto three nodes (instead of just the Tendrl node in non-HA mode). In most cases this would probably mean that etcd will get placed on two of the storage nodes in a Ceph or Gluster cluster, but we will have to validate this scenario in scale tests when we are further along. We want to make sure the disk IO impact is acceptable and that we have proper recommendations around whether disk isolation on those nodes is needed. It may be acceptable to simply colocate with the OS but on Ceph Mons for example it would probably not be a good idea to colocate with the Mon disk itself. We would also need to provide some guidance on how an HA setup is configured at the web tier. A second possible scenario in large deployments would be that we break out the performance monitoring piece onto separate hardware. This would also be for purposes of allowing the Tendrl node to focus on it's work in a large cluster (without the Graphite piece causing undue disk load). Again this is for large clusters only. The primary mode will be a self contained Tendrl node with all the necessary pieces (for small/medium clusters). Thanks Jeff On Wed, May 10, 2017 at 9:19 AM, Kenneth Hartsoe wrote: > Hi Martin, > > Thanks for discussing Console 3.0 ansible requirements and the downstream > documentation. Per our conversation, my understanding is that there is no > ansible support (user tasks) for the downstream documentation for both the > aplha and beta delivery drops, and that ansible support for downstream > documentation at GA is still TBD, after feedback received during the beta > testing cycle. > > Hi Jeff...pls let me know if this understanding is not in accordance with > your delivery plans/expectations. Thanks. > > Ken Hartsoe > Content Strategist > Red Hat Storage Documentation > > khartsoe at redhat.com; IRC: khartsoe > Office: 919 754 4770; Internal: 814 4770 > > ----- Original Message ----- > | Hi Ken, > | > | On 04/13/2017 04:57 PM, Kenneth Hartsoe wrote: > | > Looking for ansible playbooks to glean install method for inclusion in > the > | > RHS-C 3 doc. Please forward link, etc. Thank you. > | > | there is no downstream ansible based installation for now. There are 2 > | upstream ones: one used by > | Jeff and other one use and made by QE. There is agreement that > installation > | process will be more > | clear after end of Beta and there will be some kind of merge from both > | repositories to > | tendrl-ansible repository with ansible playbooks for Tendrl installation. > | > | > | -- > | Best Regards, > | Martin Kudlej. > | RHSC/USM Senior Quality Assurance Engineer > | Red Hat Czech s.r.o. > | > | Phone: +420 532 294 155 > | E-mail:mkudlej at redhat.com > | IRC: mkudlej at #brno, #gluster, #storage-qa, #rhs, #rh-ceph, > #usm-meeting > | @ redhat > | #tendrl-devel @ freenode > | > | _______________________________________________ > | Tendrl-devel mailing list > | Tendrl-devel at redhat.com > | https://www.redhat.com/mailman/listinfo/tendrl-devel > | > -- Jeff Applewhite Principal Product Manager From mkudlej at redhat.com Thu May 11 09:55:07 2017 From: mkudlej at redhat.com (Martin Kudlej) Date: Thu, 11 May 2017 11:55:07 +0200 Subject: [Tendrl-devel] Logging issue Message-ID: <1cc1187e-0cc1-512c-c431-2ae877e4b6d5@redhat.com> Hi all, I don't see any logs for node-agent. I think that probably logging configuration has changed without wiki changes. I would like to ask before I file issue. Has logging configuration changed? Now my logging configuration looks like this: $ cat /etc/tendrl/node-agent/node-agent_logging.yaml version: 1 disable_existing_loggers: False handlers: console: class: logging.StreamHandler level: DEBUG stream: ext://sys.stdout info_file_handler: class: logging.handlers.SysLogHandler facility: local5 address: /dev/log level: INFO loggers: my_module: level: ERROR handlers: [console] propagate: no root: level: INFO handlers: [console, info_file_handler] -- Best Regards, Martin Kudlej. RHSC/USM Senior Quality Assurance Engineer Red Hat Czech s.r.o. Phone: +420 532 294 155 E-mail:mkudlej at redhat.com IRC: mkudlej at #brno, #gluster, #storage-qa, #rhs, #rh-ceph, #usm-meeting @ redhat #tendrl-devel @ freenode From shtripat at redhat.com Thu May 11 10:03:13 2017 From: shtripat at redhat.com (Shubhendu Tripathi) Date: Thu, 11 May 2017 15:33:13 +0530 Subject: [Tendrl-devel] Logging issue In-Reply-To: <1cc1187e-0cc1-512c-c431-2ae877e4b6d5@redhat.com> References: <1cc1187e-0cc1-512c-c431-2ae877e4b6d5@redhat.com> Message-ID: Martin, Now all the tendrl logs are present under syslog so you can check /var/log/messages. Regards, Shubhendu On 05/11/2017 03:25 PM, Martin Kudlej wrote: > Hi all, > I don't see any logs for node-agent. I think that probably logging > configuration has changed without wiki changes. > I would like to ask before I file issue. > Has logging configuration changed? > > > Now my logging configuration looks like this: > $ cat /etc/tendrl/node-agent/node-agent_logging.yaml > version: 1 > disable_existing_loggers: False > > handlers: > console: > class: logging.StreamHandler > level: DEBUG > stream: ext://sys.stdout > > info_file_handler: > class: logging.handlers.SysLogHandler > facility: local5 > address: /dev/log > level: INFO > > loggers: > my_module: > level: ERROR > handlers: [console] > propagate: no > > root: > level: INFO > handlers: [console, info_file_handler] > From mkudlej at redhat.com Thu May 11 10:13:18 2017 From: mkudlej at redhat.com (Martin Kudlej) Date: Thu, 11 May 2017 12:13:18 +0200 Subject: [Tendrl-devel] Logging issue In-Reply-To: References: <1cc1187e-0cc1-512c-c431-2ae877e4b6d5@redhat.com> Message-ID: Hi Shubhendu, thank you for answer! On 05/11/2017 12:03 PM, Shubhendu Tripathi wrote: > Now all the tendrl logs are present under syslog so you can check /var/log/messages. Why such a change? I expect that there will be tons of logs for various tendrl components. Is there any PR where I can comments, please? -- Best Regards, Martin Kudlej. RHSC/USM Senior Quality Assurance Engineer Red Hat Czech s.r.o. Phone: +420 532 294 155 E-mail:mkudlej at redhat.com IRC: mkudlej at #brno, #gluster, #storage-qa, #rhs, #rh-ceph, #usm-meeting @ redhat #tendrl-devel @ freenode From shtripat at redhat.com Fri May 12 05:27:02 2017 From: shtripat at redhat.com (Shubhendu Tripathi) Date: Fri, 12 May 2017 10:57:02 +0530 Subject: [Tendrl-devel] Logging issue In-Reply-To: References: <1cc1187e-0cc1-512c-c431-2ae877e4b6d5@redhat.com> Message-ID: <191dffaf-404c-0f2b-ff60-60f880313ed6@redhat.com> Martin, I remember there was a specification for logging changes. Check the same at https://github.com/Tendrl/specifications/pull/127. Regards, Shubhendu On 05/11/2017 03:43 PM, Martin Kudlej wrote: > Hi Shubhendu, > > thank you for answer! > > On 05/11/2017 12:03 PM, Shubhendu Tripathi wrote: >> Now all the tendrl logs are present under syslog so you can check >> /var/log/messages. > > Why such a change? I expect that there will be tons of logs for > various tendrl components. Is there any PR where I can comments, please? > From mkudlej at redhat.com Fri May 12 05:53:59 2017 From: mkudlej at redhat.com (Martin Kudlej) Date: Fri, 12 May 2017 07:53:59 +0200 Subject: [Tendrl-devel] Logging issue In-Reply-To: <191dffaf-404c-0f2b-ff60-60f880313ed6@redhat.com> References: <1cc1187e-0cc1-512c-c431-2ae877e4b6d5@redhat.com> <191dffaf-404c-0f2b-ff60-60f880313ed6@redhat.com> Message-ID: <6fbfdaa8-9537-dd57-2065-54056552c401@redhat.com> Hi Shubhendu, thank you for response! On 05/12/2017 07:27 AM, Shubhendu Tripathi wrote: > Martin, I remember there was a specification for logging changes. > Check the same at https://github.com/Tendrl/specifications/pull/127. Maybe I'm wrong but I don't see in this specification that everything should be logged into /var/log/messages. Anyway is still any configuration option to log into separated files in addition to logging to /var/log/messages? -- Best Regards, Martin Kudlej. RHSC/USM Senior Quality Assurance Engineer Red Hat Czech s.r.o. Phone: +420 532 294 155 E-mail:mkudlej at redhat.com IRC: mkudlej at #brno, #gluster, #storage-qa, #rhs, #rh-ceph, #usm-meeting @ redhat #tendrl-devel @ freenode From asachan at redhat.com Fri May 12 06:36:45 2017 From: asachan at redhat.com (Anmol Sachan) Date: Fri, 12 May 2017 12:06:45 +0530 Subject: [Tendrl-devel] Logging issue In-Reply-To: <6fbfdaa8-9537-dd57-2065-54056552c401@redhat.com> References: <1cc1187e-0cc1-512c-c431-2ae877e4b6d5@redhat.com> <191dffaf-404c-0f2b-ff60-60f880313ed6@redhat.com> <6fbfdaa8-9537-dd57-2065-54056552c401@redhat.com> Message-ID: Hi Martin, This( https://github.com/Tendrl/specifications/pull/94 ) is another specification for logging. All the discussion is present in this specification. The related pr is https://github.com/Tendrl/node-agent/pull/212 . All the log are being pushed to syslog for which default log location is /var/log/messages. Regards, Anmol On Fri, May 12, 2017 at 11:23 AM, Martin Kudlej wrote: > Hi Shubhendu, > > thank you for response! > > On 05/12/2017 07:27 AM, Shubhendu Tripathi wrote: > >> Martin, I remember there was a specification for logging changes. >> Check the same at https://github.com/Tendrl/specifications/pull/127. >> > > Maybe I'm wrong but I don't see in this specification that everything > should be logged into /var/log/messages. > Anyway is still any configuration option to log into separated files in > addition to logging to /var/log/messages? > > -- > Best Regards, > Martin Kudlej. > RHSC/USM Senior Quality Assurance Engineer > Red Hat Czech s.r.o. > > Phone: +420 532 294 155 > E-mail:mkudlej at redhat.com > IRC: mkudlej at #brno, #gluster, #storage-qa, #rhs, #rh-ceph, > #usm-meeting @ redhat > #tendrl-devel @ freenode > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From tserong at suse.com Wed May 17 06:29:04 2017 From: tserong at suse.com (Tim Serong) Date: Wed, 17 May 2017 16:29:04 +1000 Subject: [Tendrl-devel] ceph-mgr REST API Message-ID: <6c4ce752-4594-aa31-9d0e-35985e379c5a@suse.com> Hi All, There's a PR that's been open for a while to add a new pecan-based REST API to ceph-mgr: https://github.com/ceph/ceph/pull/14457 This is intended to be consumed internally by management tools such as openATTIC[1] and Tendrl[2] (and anything else that wants to manage a Ceph cluster via a REST API). There's been quite some discussion on the PR, and we also spoke about it at the May CDM, but I don't think much mention has occurred on the various mailing lists, so I'm writing this to raise awareness and solicit feedback. There's a desire to get this PR merged for the Luminous release, but possibly marked "experimental", so that at least we have something out there that people can start using. I had volunteered to document the delta between this ceph-mgr REST API and the Ceph REST API currently present in openATTIC, to attempt to gauge the effort involved in making openATTIC use the ceph-mgr REST API, but Sebastian Wagner has beaten me to it with some good details[3] (thanks!), so I'll just try to summarise here for the record: - Both APIs provide: - For OSDs: list, get details, modify (reweight, up/in state) - For Pools: list, get details, modify, delete - A means of handling long running requests (although AIUI these work somewhat differently) - The openATTIC API provides in addition: - For PGs: list (can be filtered by pool, osd), get details - For RBD volumes: list, create, delete, modify - For CephFS: list - For EC Profiles: list, create, delete - Pagination of lists (important for non-small clusters) - The ceph-mgr REST API provides in addition: - For cluster config options: list, get value - For CRUSH rules: list - For MONs: list, get details - For OSDs: run commands (scrub, deep_scrub, repair) - For Servers: list, get details There are also some differences in naming and which fields are exposed[4], but hopefully this is enough to give a general idea. My apologies to Boris Ranto (ceph-mgr REST API) and the openATTIC folks if I've gotten anything wrong here. Regards, Tim [1] https://www.openattic.org/ [2] http://tendrl.org/ [3] https://github.com/ceph/ceph/pull/14457#issuecomment-301954323 [4] For example, compare: GET /openattic/api/ceph/80b30fd5-e0b2-363e-9ddd-63e87bac02c6/osds/0 { "id": 0, "crush_weight": 0.018494, "exists": 1, "name": "osd.0", "primary_affinity": 1.0, "reweight": 1.0, "status": "up", "type": "osd", "hostname": "ses4-5", "in_state": 1, "kb": 19911660, "kb_used": 35792, "kb_avail": 19875868 } GET https://localhost:8002/osd/0 { "cluster_addr": "192.168.12.150:6801/1861", "down_at": 29, "heartbeat_back_addr": "192.168.12.150:6802/1861", "heartbeat_front_addr": "192.168.12.150:6803/1861", "in": 1, "last_clean_begin": 6, "last_clean_end": 26, "lost_at": 0, "osd": 0, "pools": [ 0 ], "primary_affinity": 1.0, "public_addr": "192.168.12.150:6800/1861", "reweight": 1.0, "server": "ses4-5", "state": [ "exists", "up" ], "up": 1, "up_from": 30, "up_thru": 42, "uuid": "fd129a52-0448-4e04-b41d-680e796f2731", "valid_commands": [ "scrub", "deep_scrub", "repair" ], "weight": 1.0 } -- Tim Serong Senior Clustering Engineer SUSE tserong at suse.com From mbukatov at redhat.com Thu May 18 08:21:00 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Thu, 18 May 2017 10:21:00 +0200 Subject: [Tendrl-devel] extraneous packages in tendrl dependencies repository Message-ID: Dear all, during review of upstream rpm validation tests we noticed issues with packages: * rubygem-sinatra * rubygem-sinatra-doc * rubygem-tilt * rubygem-tilt-doc See full report here: https://ci.centos.org/view/Tendrl/job/tendrl-0-2-package-validation-rpmdeplint/292/testReport/ As this log line shows: ``` [06:08:34,097] [ FAIL ] test_rpm:: rubygem-sinatra-1.4.5-1.el7.centos.noarch would be upgraded by rubygem-sinatra-1:1.4.8-2.el7.noarch from repo fedora-epel ``` We have package rubygem-sinatra-1.4.5-1.el7.centos.noarch in tendrl dependencies repository[1], while epel7 contains already rubygem-sinatra-1:1.4.8-2.el7.noarch[2] This shows that we are rebuilding package, which is already available in epel, and our version is older - which means that it would not be used as epel one would be installed as an update when yum update is run. For this reason, we should drop all affected packages from tendrl dependencies repository. [1] https://copr.fedorainfracloud.org/coprs/tendrl/dependencies/build/538791/ [2] https://koji.fedoraproject.org/koji/buildinfo?buildID=884515 -- Martin Bukatovic USM QE team Red Hat From rkanade at redhat.com Thu May 18 17:59:02 2017 From: rkanade at redhat.com (Rohan Kanade) Date: Thu, 18 May 2017 23:29:02 +0530 Subject: [Tendrl-devel] tendrl-release v1.3.0 (beta) is available Message-ID: Hello, The Tendrl team is happy to present release v1.3.0 Packages: tendrl-commons v1.3.0 tendrl-node-agent v1.3.0 tendrl-ceph-integration v1.3.0 tendrl-gluster-integration v1.3.0 tendrl-performance-monitoring v1.3.0 tendrl-node-monitoring v1.3.0 tendrl-alerting v1.3.0 tendrl-api v1.3.0 tendrl-dashboard v1.3.0 Release wiki: https://github.com/Tendrl/documentation/wiki/Tendrl-release-v1.3.0B Cheers! From jefbrown at redhat.com Thu May 18 18:03:37 2017 From: jefbrown at redhat.com (Jeff Brown) Date: Thu, 18 May 2017 14:03:37 -0400 Subject: [Tendrl-devel] tendrl-release v1.3.0 (beta) is available In-Reply-To: References: Message-ID: Congratulations. Can you send a link to the sandbox as well? (with the password?) Thanks On Thu, May 18, 2017 at 1:59 PM, Rohan Kanade wrote: > Hello, > > The Tendrl team is happy to present release v1.3.0 > > Packages: > tendrl-commons v1.3.0 > tendrl-node-agent v1.3.0 > tendrl-ceph-integration v1.3.0 > tendrl-gluster-integration v1.3.0 > tendrl-performance-monitoring v1.3.0 > tendrl-node-monitoring v1.3.0 > tendrl-alerting v1.3.0 > tendrl-api v1.3.0 > tendrl-dashboard v1.3.0 > > > Release wiki: > https://github.com/Tendrl/documentation/wiki/Tendrl-release-v1.3.0B > > Cheers! > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From sankarshan at redhat.com Thu May 18 18:04:42 2017 From: sankarshan at redhat.com (sankarshan) Date: Thu, 18 May 2017 23:34:42 +0530 Subject: [Tendrl-devel] tendrl-release v1.3.0 (beta) is available In-Reply-To: References: Message-ID: On 18 May 2017 at 23:33, Jeff Brown wrote: > Congratulations. Can you send a link to the sandbox as well? (with the > password?) > Jeff, the sand-boxes are slated to be refreshed through tomorrow (India time) and we'll update the access detail once we are set. From japplewh at redhat.com Thu May 18 18:16:35 2017 From: japplewh at redhat.com (Jeff Applewhite) Date: Thu, 18 May 2017 14:16:35 -0400 Subject: [Tendrl-devel] tendrl-release v1.3.0 (beta) is available In-Reply-To: References: Message-ID: Congratulations and thanks to everyone contributed! ~Jeff On Thu, May 18, 2017 at 2:04 PM, sankarshan wrote: > On 18 May 2017 at 23:33, Jeff Brown wrote: > > Congratulations. Can you send a link to the sandbox as well? (with the > > password?) > > > > Jeff, the sand-boxes are slated to be refreshed through tomorrow > (India time) and we'll update the access detail once we are set. > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > -- Jeff Applewhite Principal Product Manager From jefbrown at redhat.com Thu May 18 18:17:17 2017 From: jefbrown at redhat.com (Jeff Brown) Date: Thu, 18 May 2017 14:17:17 -0400 Subject: [Tendrl-devel] tendrl-release v1.3.0 (beta) is available In-Reply-To: References: Message-ID: Thanks On Thu, May 18, 2017 at 2:04 PM, sankarshan wrote: > On 18 May 2017 at 23:33, Jeff Brown wrote: > > Congratulations. Can you send a link to the sandbox as well? (with the > > password?) > > > > Jeff, the sand-boxes are slated to be refreshed through tomorrow > (India time) and we'll update the access detail once we are set. > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From julim at redhat.com Thu May 18 20:24:52 2017 From: julim at redhat.com (Ju Lim) Date: Thu, 18 May 2017 16:24:52 -0400 Subject: [Tendrl-devel] tendrl-release v1.3.0 (beta) is available In-Reply-To: References: Message-ID: Congrats to everyone in getting out the beta! Is there any plans to do a demo? I saw mention of it in the Program Call minutes, but I've not yet seen an invite. Please advie. Thanks, Ju On Thu, May 18, 2017 at 2:17 PM, Jeff Brown wrote: > Thanks > > > On Thu, May 18, 2017 at 2:04 PM, sankarshan wrote: > > > On 18 May 2017 at 23:33, Jeff Brown wrote: > > > Congratulations. Can you send a link to the sandbox as well? (with > the > > > password?) > > > > > > > Jeff, the sand-boxes are slated to be refreshed through tomorrow > > (India time) and we'll update the access detail once we are set. > > > > _______________________________________________ > > Tendrl-devel mailing list > > Tendrl-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/tendrl-devel > > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > -- Ju Lim Red Hat Office: 978-399-0422 Mobile: 781-507-1323 Email: julim at redhat.com IRC: julim From mbukatov at redhat.com Mon May 22 08:52:01 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Mon, 22 May 2017 10:52:01 +0200 Subject: [Tendrl-devel] extraneous packages in tendrl dependencies repository In-Reply-To: References: Message-ID: <6c43e4c6-6ee9-ee78-9812-c3d629ec69cb@redhat.com> On 05/18/2017 10:21 AM, Martin Bukatovic wrote: > Dear all, > > during review of upstream rpm validation tests we noticed issues with > packages: > > * rubygem-sinatra > * rubygem-sinatra-doc > * rubygem-tilt > * rubygem-tilt-doc > > See full report here: > > https://ci.centos.org/view/Tendrl/job/tendrl-0-2-package-validation-rpmdeplint/292/testReport/ > > As this log line shows: > > ``` > [06:08:34,097] [ FAIL ] test_rpm:: > rubygem-sinatra-1.4.5-1.el7.centos.noarch would be upgraded by > rubygem-sinatra-1:1.4.8-2.el7.noarch from repo fedora-epel > ``` > > We have package rubygem-sinatra-1.4.5-1.el7.centos.noarch in > tendrl dependencies repository[1], while epel7 contains already > rubygem-sinatra-1:1.4.8-2.el7.noarch[2] > > This shows that we are rebuilding package, which is already > available in epel, and our version is older - which means that > it would not be used as epel one would be installed as an update > when yum update is run. > > For this reason, we should drop all affected packages from tendrl > dependencies repository. > > [1] > https://copr.fedorainfracloud.org/coprs/tendrl/dependencies/build/538791/ > [2] https://koji.fedoraproject.org/koji/buildinfo?buildID=884515 ping - as of today, I still see the packages in the repository Do we have some reason to require particular versions of given packages? If yes, this is a problem. -- Martin Bukatovic USM QE team Red Hat From mbukatov at redhat.com Mon May 22 09:14:25 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Mon, 22 May 2017 11:14:25 +0200 Subject: [Tendrl-devel] ceph-mgr REST API In-Reply-To: <6c4ce752-4594-aa31-9d0e-35985e379c5a@suse.com> References: <6c4ce752-4594-aa31-9d0e-35985e379c5a@suse.com> Message-ID: <582e4a87-0c93-05f9-e43c-cb2614d8daba@redhat.com> On 05/17/2017 08:29 AM, Tim Serong wrote: > There's a PR that's been open for a while to add a new pecan-based REST > API to ceph-mgr: > > https://github.com/ceph/ceph/pull/14457 > > This is intended to be consumed internally by management tools such as > openATTIC[1] and Tendrl[2] (and anything else that wants to manage a > Ceph cluster via a REST API). > > There's been quite some discussion on the PR, and we also spoke about it > at the May CDM, but I don't think much mention has occurred on the > various mailing lists, so I'm writing this to raise awareness and > solicit feedback. > > There's a desire to get this PR merged for the Luminous release, but > possibly marked "experimental", so that at least we have something out > there that people can start using. > > I had volunteered to document the delta between this ceph-mgr REST API > and the Ceph REST API currently present in openATTIC, to attempt to > gauge the effort involved in making openATTIC use the ceph-mgr REST API, > but Sebastian Wagner has beaten me to it with some good details[3] > (thanks!), so I'll just try to summarise here for the record: > > - Both APIs provide: > - For OSDs: list, get details, modify (reweight, up/in state) > - For Pools: list, get details, modify, delete > - A means of handling long running requests (although AIUI these > work somewhat differently) > > - The openATTIC API provides in addition: > - For PGs: list (can be filtered by pool, osd), get details > - For RBD volumes: list, create, delete, modify > - For CephFS: list > - For EC Profiles: list, create, delete > - Pagination of lists (important for non-small clusters) > > - The ceph-mgr REST API provides in addition: > - For cluster config options: list, get value > - For CRUSH rules: list > - For MONs: list, get details > - For OSDs: run commands (scrub, deep_scrub, repair) > - For Servers: list, get details > > There are also some differences in naming and which fields are > exposed[4], but hopefully this is enough to give a general idea. My > apologies to Boris Ranto (ceph-mgr REST API) and the openATTIC folks if > I've gotten anything wrong here. Thanks for keeping us involved. My feeling is that we should try to evaluate Tendrl API and try to keep as close as possible to proposed ceph-mgr API - unless we have some problem with the design, but then we should try to influence the design on ceph side. It seems that we need to cleanup the Tenrl API anyway, as the current implementation suffers with some design problems - see email "[Tendrl-devel] REST API consistency". -- Martin Bukatovic USM QE team Red Hat From nthomas at redhat.com Mon May 22 12:37:13 2017 From: nthomas at redhat.com (Nishanth Thomas) Date: Mon, 22 May 2017 18:07:13 +0530 Subject: [Tendrl-devel] extraneous packages in tendrl dependencies repository In-Reply-To: <6c43e4c6-6ee9-ee78-9812-c3d629ec69cb@redhat.com> References: <6c43e4c6-6ee9-ee78-9812-c3d629ec69cb@redhat.com> Message-ID: Hi Martin, Thanks for pointing this out Earlier these packages were not available in epel - this got added recently. There is specifically no reason to pin down to that old version and we are making the changes to fix it. Thanks, Nishanth On Mon, May 22, 2017 at 2:22 PM, Martin Bukatovic wrote: > On 05/18/2017 10:21 AM, Martin Bukatovic wrote: > > Dear all, > > > > during review of upstream rpm validation tests we noticed issues with > > packages: > > > > * rubygem-sinatra > > * rubygem-sinatra-doc > > * rubygem-tilt > > * rubygem-tilt-doc > > > > See full report here: > > > > https://ci.centos.org/view/Tendrl/job/tendrl-0-2-package- > validation-rpmdeplint/292/testReport/ > > > > As this log line shows: > > > > ``` > > [06:08:34,097] [ FAIL ] test_rpm:: > > rubygem-sinatra-1.4.5-1.el7.centos.noarch would be upgraded by > > rubygem-sinatra-1:1.4.8-2.el7.noarch from repo fedora-epel > > ``` > > > > We have package rubygem-sinatra-1.4.5-1.el7.centos.noarch in > > tendrl dependencies repository[1], while epel7 contains already > > rubygem-sinatra-1:1.4.8-2.el7.noarch[2] > > > > This shows that we are rebuilding package, which is already > > available in epel, and our version is older - which means that > > it would not be used as epel one would be installed as an update > > when yum update is run. > > > > For this reason, we should drop all affected packages from tendrl > > dependencies repository. > > > > [1] > > https://copr.fedorainfracloud.org/coprs/tendrl/dependencies/ > build/538791/ > > [2] https://koji.fedoraproject.org/koji/buildinfo?buildID=884515 > > ping - as of today, I still see the packages in the repository > > Do we have some reason to require particular versions of given > packages? If yes, this is a problem. > > > -- > Martin Bukatovic > USM QE team > Red Hat > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From sankarshan.mukhopadhyay at gmail.com Tue May 23 02:41:08 2017 From: sankarshan.mukhopadhyay at gmail.com (Sankarshan Mukhopadhyay) Date: Tue, 23 May 2017 08:11:08 +0530 Subject: [Tendrl-devel] Ansible installation methods and playbooks for Tendrl Message-ID: The topic has been brought up a couple of times. Yesterday, Jeff Applewhite facilitated a conversation which has resulted in the following points of follow up action + the current of playbooks in the usmqe repository would be looked into to get roles set up correctly + the conversation around roles is being tracked at + the playbooks will be made available through the tendrl-ansible repository, preferably as a new branch + once these are available; the developers would be testing the deployment with these against the current builds + the playbooks will continue to receive contributions from all Tendrl contributors and would thus form the default installation method recommended by Tendrl Please append/amend as appropriate. -- sankarshan mukhopadhyay From sankarshan.mukhopadhyay at gmail.com Tue May 23 02:51:38 2017 From: sankarshan.mukhopadhyay at gmail.com (Sankarshan Mukhopadhyay) Date: Tue, 23 May 2017 08:21:38 +0530 Subject: [Tendrl-devel] [Feedback] Tendrl documentation ToC Message-ID: A few weeks back the conversation about (re)structuring the existing written content around Tendrl came up. has a proposed Table of Content The existing documentation does not cover *all* of the ToC. Thus, there is net new content which needs to be written down - particularly around the topics of monitoring and debugging. -- sankarshan mukhopadhyay From mrugesh at brainfunked.org Tue May 23 15:18:51 2017 From: mrugesh at brainfunked.org (Mrugesh Karnik) Date: Tue, 23 May 2017 20:48:51 +0530 Subject: [Tendrl-devel] Ansible installation methods and playbooks for Tendrl In-Reply-To: References: Message-ID: On 23 May 2017 at 08:11, Sankarshan Mukhopadhyay wrote: > + the playbooks will be made available through the tendrl-ansible > repository, preferably as a new branch > > + once these are available; the developers would be testing the > deployment with these against the current builds > > + the playbooks will continue to receive contributions from all Tendrl > contributors and would thus form the default installation method > recommended by Tendrl > > Please append/amend as appropriate. I'm looking through the various deployment related sanity checks and the ansible playbooks this week. I'll look at what gaps exist between the two sets of playbooks. I currently believe that the branch drawn up on tendrl-ansible based on the usmqe playbooks should represent only the gaps filled in, rather than a wholesale update. In any case, if the usmqe playbooks are a substantial update, we can look at replacing the tendrl-ansible playbooks with them. -- Mrugesh From mbukatov at redhat.com Tue May 23 17:18:07 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Tue, 23 May 2017 19:18:07 +0200 Subject: [Tendrl-devel] rpm nvr for snapshot builds Message-ID: <9a132081-eb29-3dc1-5439-765628f3d828@redhat.com> Dear all, the package names for tendrl (non release) snapshot builds uses invalid version scheme. When you look into tendrl/tendrl[1] (or tendrl/tendrl-develop) copr, you will see builds named like: tendrl-alerting 1.3.0-05_23_2017_02_49_04 which breaks Fedora packaging guidelines for packaging snapshot builds[2] and creates problem with pairing the build with commit/code used to create it. If we decide to solve this, we should do this by adopting the fedora guidelines, which states: ~~~ All snapshots MUST contain a snapshot information field () in the Release: tag. That field must at minimum consist of the date in eight-digit "YYYYMMDD" format. The packager MAY include up to 17 characters of additional information after the date. The following formats are suggested: YYYYMMDD. YYYYMMDD Where is a short string identifying the source code control system upstream uses (e.g. "git", "svn", "hg") or the string "snap". is either a short git commit hash, a subversion revision number, or something else useful in identifying the precise revision in upstream's source code control system. Obviously if CVS is used, no such revision information exists, so it would be omitted, but otherwise it SHOULD be included. ~~~ Examples could be found on fedora wiki[3] On the other hand, if we don't care about tracking relationship between code and snapshot builds, we don't have not change anything. The versions in snapshot repositories would continue to look weird and useless, but as long as that is the expectation, it should work. Running rpmlint and other CI checks would be possible as it works right now. [1] https://copr.fedorainfracloud.org/coprs/tendrl/tendrl/packages/ [2] https://fedoraproject.org/wiki/Packaging:Versioning [3] https://fedoraproject.org/wiki/Package_Versioning_Examples#Complex_versioning_examples -- Martin Bukatovic USM QE team Red Hat From kdreyer at redhat.com Tue May 23 18:21:59 2017 From: kdreyer at redhat.com (Ken Dreyer) Date: Tue, 23 May 2017 12:21:59 -0600 Subject: [Tendrl-devel] rpm nvr for snapshot builds In-Reply-To: <9a132081-eb29-3dc1-5439-765628f3d828@redhat.com> References: <9a132081-eb29-3dc1-5439-765628f3d828@redhat.com> Message-ID: On Tue, May 23, 2017 at 11:18 AM, Martin Bukatovic wrote: > > which breaks Fedora packaging guidelines for packaging snapshot > builds[2] and creates problem with pairing the build with > commit/code used to create it. Are these new Version-Releases generated regardless of whether there are any code changes? - Ken From nthomas at redhat.com Tue May 23 22:34:35 2017 From: nthomas at redhat.com (Nishanth Thomas) Date: Wed, 24 May 2017 04:04:35 +0530 Subject: [Tendrl-devel] Issue with tendrl release repository on copr Message-ID: All, There is an ongoing issue with tendrl release repository on copr[1] where the repo data is not updated with latest packages. Due to this, if you add the repo and do a normal install, older rpms are served which results in undesirable behaviour. So it is advised to provide the full URL(For example: yum install https://copr-be.cloud.fedoraproject.org/results/tendrl/release/epel-7-x86_64/00555871-tendrl-commons/tendrl-commons-1.3.0-1.el7.centos.noarch.rpm) while installing tendrl packages from release repository until this issue is resolved Thanks, Nishanth [1]https://copr.fedorainfracloud.org/coprs/tendrl/release/ From sankarshan.mukhopadhyay at gmail.com Wed May 24 05:57:08 2017 From: sankarshan.mukhopadhyay at gmail.com (Sankarshan Mukhopadhyay) Date: Wed, 24 May 2017 11:27:08 +0530 Subject: [Tendrl-devel] rpm nvr for snapshot builds In-Reply-To: <9a132081-eb29-3dc1-5439-765628f3d828@redhat.com> References: <9a132081-eb29-3dc1-5439-765628f3d828@redhat.com> Message-ID: On Tue, May 23, 2017 at 10:48 PM, Martin Bukatovic wrote: > the package names for tendrl (non release) snapshot builds > uses invalid version scheme. When you look into tendrl/tendrl[1] > (or tendrl/tendrl-develop) copr, you will see builds named like: > > tendrl-alerting 1.3.0-05_23_2017_02_49_04 > > which breaks Fedora packaging guidelines for packaging snapshot > builds[2] and creates problem with pairing the build with > commit/code used to create it. > > If we decide to solve this, we should do this by adopting the > fedora guidelines, which states: > Would it be possible for you to send a PR that helps us adopt the guidelines? -- sankarshan mukhopadhyay From mbukatov at redhat.com Wed May 24 08:16:07 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Wed, 24 May 2017 10:16:07 +0200 Subject: [Tendrl-devel] rpm nvr for snapshot builds In-Reply-To: References: <9a132081-eb29-3dc1-5439-765628f3d828@redhat.com> Message-ID: <91b506e2-1162-ebe0-fd5e-04337f7fa576@redhat.com> On 05/23/2017 08:21 PM, Ken Dreyer wrote: > On Tue, May 23, 2017 at 11:18 AM, Martin Bukatovic wrote: >> >> which breaks Fedora packaging guidelines for packaging snapshot >> builds[2] and creates problem with pairing the build with >> commit/code used to create it. > > Are these new Version-Releases generated regardless of whether there > are any code changes? Right now, packages in tendrl/tendrl (and tendrl-develop) copr are created no matter if the same version has been build previously, as there is no information which commit the build is based on. If we change the workflow and include commit id into release information, we can decide not to rebuild the package if nothing has changed - if that would suit our needs better. -- Martin Bukatovic USM QE team Red Hat From mbukatov at redhat.com Wed May 24 08:16:58 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Wed, 24 May 2017 10:16:58 +0200 Subject: [Tendrl-devel] rpm nvr for snapshot builds In-Reply-To: References: <9a132081-eb29-3dc1-5439-765628f3d828@redhat.com> Message-ID: On 05/24/2017 07:57 AM, Sankarshan Mukhopadhyay wrote: > On Tue, May 23, 2017 at 10:48 PM, Martin Bukatovic wrote: > >> the package names for tendrl (non release) snapshot builds >> uses invalid version scheme. When you look into tendrl/tendrl[1] >> (or tendrl/tendrl-develop) copr, you will see builds named like: >> >> tendrl-alerting 1.3.0-05_23_2017_02_49_04 >> >> which breaks Fedora packaging guidelines for packaging snapshot >> builds[2] and creates problem with pairing the build with >> commit/code used to create it. >> >> If we decide to solve this, we should do this by adopting the >> fedora guidelines, which states: >> > > Would it be possible for you to send a PR that helps us adopt the guidelines? Could you point me to the related documentation or code? -- Martin Bukatovic USM QE team Red Hat From mbukatov at redhat.com Wed May 24 08:19:37 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Wed, 24 May 2017 10:19:37 +0200 Subject: [Tendrl-devel] Fwd: Re: [Feedback] Tendrl documentation ToC In-Reply-To: <5bec62f1-08a3-e09f-b7ef-4a40b51d9c78@redhat.com> References: <5bec62f1-08a3-e09f-b7ef-4a40b51d9c78@redhat.com> Message-ID: <7a6be082-73b5-7938-c695-8728a18a84b9@redhat.com> -------- Forwarded Message -------- Subject: Re: [Tendrl-devel] [Feedback] Tendrl documentation ToC Date: Tue, 23 May 2017 13:55:41 +0200 From: Martin Bukatovic Organization: Red Hat To: Sankarshan Mukhopadhyay On 05/23/2017 04:51 AM, Sankarshan Mukhopadhyay wrote: > A few weeks back the conversation about (re)structuring the existing > written content around Tendrl came up. > > > has a proposed Table of Content > > The existing documentation does not cover *all* of the ToC. Thus, > there is net new content which needs to be written down - particularly > around the topics of monitoring and debugging. The TOC looks good, at first sight I feel that the following topics related to the ones already mentioned in your toc needs to be expanded, which will influence the structure significantly: * deployment architectures/scenarios - and suggestions when to use particular setup/configuration - while this seems to be included in "Architecture of Tendrl" from the TOC, it has impact on "Getting Started" section as well (eg. description of installation steps for each deployment type). * hw/cluster requirements * glossary I agree with you that there is a room for improvement: we have multiple sources of documentation at this point: * docs directory of particular components * tendrl-documentation repository * tendrl-documentation github wiki * and your new toc page somewhere else In each case, a different syntax/documentation system is used and some documents are already not up to date to current plans/implementation. While we improved the docs since I reported issue https://github.com/Tendrl/specifications/issues/25 we haven't fully resolved the clarification part, nor we enforce anything (eg. there are no ascidoc builds for tendrl-documentation repository). -- Martin Bukatovic USM QE team Red Hat From shtripat at redhat.com Wed May 24 08:24:32 2017 From: shtripat at redhat.com (Shubhendu Tripathi) Date: Wed, 24 May 2017 13:54:32 +0530 Subject: [Tendrl-devel] Fwd: Re: [Feedback] Tendrl documentation ToC In-Reply-To: <7a6be082-73b5-7938-c695-8728a18a84b9@redhat.com> References: <5bec62f1-08a3-e09f-b7ef-4a40b51d9c78@redhat.com> <7a6be082-73b5-7938-c695-8728a18a84b9@redhat.com> Message-ID: <98b27db2-a5af-d903-1750-9ec8e90d0228@redhat.com> I feel some section (appendix may be) for helping user to create objects/files, loading CPU/Memory etc in the cluster for changing the utilization would be good. This would help verifying the utilization details in dashboards. Regards, Shubhendu On 05/24/2017 01:49 PM, Martin Bukatovic wrote: > > > -------- Forwarded Message -------- > Subject: Re: [Tendrl-devel] [Feedback] Tendrl documentation ToC > Date: Tue, 23 May 2017 13:55:41 +0200 > From: Martin Bukatovic > Organization: Red Hat > To: Sankarshan Mukhopadhyay > > On 05/23/2017 04:51 AM, Sankarshan Mukhopadhyay wrote: >> A few weeks back the conversation about (re)structuring the existing >> written content around Tendrl came up. >> >> >> has a proposed Table of Content >> >> The existing documentation does not cover *all* of the ToC. Thus, >> there is net new content which needs to be written down - particularly >> around the topics of monitoring and debugging. > The TOC looks good, at first sight I feel that the following topics > related to the ones already mentioned in your toc needs to be expanded, > which will influence the structure significantly: > > * deployment architectures/scenarios - and suggestions when to use > particular setup/configuration - while this seems to be included > in "Architecture of Tendrl" from the TOC, it has impact on "Getting > Started" section as well (eg. description of installation steps for > each deployment type). > * hw/cluster requirements > * glossary > > I agree with you that there is a room for improvement: we have > multiple sources of documentation at this point: > > * docs directory of particular components > * tendrl-documentation repository > * tendrl-documentation github wiki > * and your new toc page somewhere else > > In each case, a different syntax/documentation system is used > and some documents are already not up to date to current > plans/implementation. While we improved the docs since I > reported issue https://github.com/Tendrl/specifications/issues/25 > we haven't fully resolved the clarification part, nor we enforce > anything (eg. there are no ascidoc builds for tendrl-documentation > repository). > From kdreyer at redhat.com Wed May 24 16:40:10 2017 From: kdreyer at redhat.com (Ken Dreyer) Date: Wed, 24 May 2017 10:40:10 -0600 Subject: [Tendrl-devel] rpm nvr for snapshot builds In-Reply-To: <91b506e2-1162-ebe0-fd5e-04337f7fa576@redhat.com> References: <9a132081-eb29-3dc1-5439-765628f3d828@redhat.com> <91b506e2-1162-ebe0-fd5e-04337f7fa576@redhat.com> Message-ID: On Wed, May 24, 2017 at 2:16 AM, Martin Bukatovic wrote: > > If we change the workflow and include commit id into release > information, we can decide not to rebuild the package if nothing > has changed - if that would suit our needs better. Definitely +1 to this. That's what "make snapshot" is supposed to do. If the Jenkins jobs used a pollscm trigger rather than a basic timer, then they would only kick off if Git has changed. - Ken From nthomas at redhat.com Wed May 24 16:44:58 2017 From: nthomas at redhat.com (Nishanth Thomas) Date: Wed, 24 May 2017 22:14:58 +0530 Subject: [Tendrl-devel] Tendrl package status Message-ID: All, The latest tendrl packages containing the fixes for the issues listed below: https://github.com/Tendrl/dashboard/issues/335 https://github.com/Tendrl/dashboard/issues/361 https://github.com/Tendrl/node-agent/issues/453 https://github.com/Tendrl/dashboard/issues/353 https://github.com/Tendrl/api/issues/179 https://github.com/Tendrl/api/issues/141 https://github.com/Tendrl/node-agent/issues/414 https://github.com/Tendrl/dashboard/issues/367 are now available. Further information; Latest packages: https://copr.fedorainfracloud.org/coprs/tendrl/release/packages/ Latest dependencies: https://copr.fedorainfracloud.org/coprs/tendrl/dependencies/packages/ Install: https://github.com/Tendrl/documentation/wiki/Tendrl-Package-Installation-Reference Thanks, Nishanth From sankarshan.mukhopadhyay at gmail.com Wed May 24 16:53:23 2017 From: sankarshan.mukhopadhyay at gmail.com (Sankarshan Mukhopadhyay) Date: Wed, 24 May 2017 22:23:23 +0530 Subject: [Tendrl-devel] rpm nvr for snapshot builds In-Reply-To: References: <9a132081-eb29-3dc1-5439-765628f3d828@redhat.com> Message-ID: On Wed, May 24, 2017 at 1:46 PM, Martin Bukatovic wrote: > On 05/24/2017 07:57 AM, Sankarshan Mukhopadhyay wrote: >> On Tue, May 23, 2017 at 10:48 PM, Martin Bukatovic wrote: >> >>> the package names for tendrl (non release) snapshot builds >>> uses invalid version scheme. When you look into tendrl/tendrl[1] >>> (or tendrl/tendrl-develop) copr, you will see builds named like: >>> >>> tendrl-alerting 1.3.0-05_23_2017_02_49_04 >>> >>> which breaks Fedora packaging guidelines for packaging snapshot >>> builds[2] and creates problem with pairing the build with >>> commit/code used to create it. >>> >>> If we decide to solve this, we should do this by adopting the >>> fedora guidelines, which states: >>> >> >> Would it be possible for you to send a PR that helps us adopt the guidelines? > > Could you point me to the related documentation or code? > Ok. Let's do this - we work towards giving you access etc to the Jenkins instance invoking the jobs off Copr and hopefully that will enable us to move in the direction you have put across. -- sankarshan mukhopadhyay From kdreyer at redhat.com Wed May 24 18:00:13 2017 From: kdreyer at redhat.com (Ken Dreyer) Date: Wed, 24 May 2017 12:00:13 -0600 Subject: [Tendrl-devel] Issue with tendrl release repository on copr In-Reply-To: References: Message-ID: Hi Nishanth, What is the url to the ticket where this was reported to the Copr team? - Ken On Tue, May 23, 2017 at 4:34 PM, Nishanth Thomas wrote: > All, > > There is an ongoing issue with tendrl release repository on copr[1] where > the repo data is not updated with latest packages. Due to this, if you add > the repo and do a normal install, older rpms are served which results in > undesirable behaviour. So it is advised to provide the full URL(For > example: yum install > https://copr-be.cloud.fedoraproject.org/results/tendrl/release/epel-7-x86_64/00555871-tendrl-commons/tendrl-commons-1.3.0-1.el7.centos.noarch.rpm) > while installing tendrl packages from release repository until this issue > is resolved > > Thanks, > Nishanth > [1]https://copr.fedorainfracloud.org/coprs/tendrl/release/ > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From mkudlej at redhat.com Thu May 25 08:40:18 2017 From: mkudlej at redhat.com (Martin Kudlej) Date: Thu, 25 May 2017 10:40:18 +0200 Subject: [Tendrl-devel] rpm nvr for snapshot builds In-Reply-To: References: <9a132081-eb29-3dc1-5439-765628f3d828@redhat.com> Message-ID: <3cb2022a-7a4a-d8bd-aa88-d56964d19eaf@redhat.com> Hi Sankarshan, On 05/24/2017 06:53 PM, Sankarshan Mukhopadhyay wrote: > On Wed, May 24, 2017 at 1:46 PM, Martin Bukatovic wrote: >> On 05/24/2017 07:57 AM, Sankarshan Mukhopadhyay wrote: >>> On Tue, May 23, 2017 at 10:48 PM, Martin Bukatovic wrote: >>> >>>> the package names for tendrl (non release) snapshot builds >>>> uses invalid version scheme. When you look into tendrl/tendrl[1] >>>> (or tendrl/tendrl-develop) copr, you will see builds named like: >>>> >>>> tendrl-alerting 1.3.0-05_23_2017_02_49_04 >>>> >>>> which breaks Fedora packaging guidelines for packaging snapshot >>>> builds[2] and creates problem with pairing the build with >>>> commit/code used to create it. >>>> >>>> If we decide to solve this, we should do this by adopting the >>>> fedora guidelines, which states: >>>> >>> >>> Would it be possible for you to send a PR that helps us adopt the guidelines? >> >> Could you point me to the related documentation or code? >> > > Ok. Let's do this - we work towards giving you access etc to the > Jenkins instance invoking the jobs off Copr and hopefully that will > enable us to move in the direction you have put across. I think the best solution for this is to move those jobs into git repository(Jenkins job builder format) and use them in ci.centos.org. It will enhance CI workflow. There is already proposal for some time for this https://github.com/Tendrl/usmqe-centos-ci/pull/1 -- Best Regards, Martin Kudlej. RHSC/USM Senior Quality Assurance Engineer Red Hat Czech s.r.o. Phone: +420 532 294 155 E-mail:mkudlej at redhat.com IRC: mkudlej at #brno, #gluster, #storage-qa, #rhs, #rh-ceph, #usm-meeting @ redhat #tendrl-devel @ freenode From sankarshan.mukhopadhyay at gmail.com Thu May 25 16:01:08 2017 From: sankarshan.mukhopadhyay at gmail.com (Sankarshan Mukhopadhyay) Date: Thu, 25 May 2017 21:31:08 +0530 Subject: [Tendrl-devel] [Feedback] Tendrl documentation ToC In-Reply-To: References: Message-ID: Great feedback on this thread. Thank you! I'll consolidate it over the weekend and make a stub entry on the Github wiki - the reason to use the hackmd pad was that my 2FA was causing issues. On Tue, May 23, 2017 at 8:21 AM, Sankarshan Mukhopadhyay wrote: > A few weeks back the conversation about (re)structuring the existing > written content around Tendrl came up. > > > has a proposed Table of Content > > The existing documentation does not cover *all* of the ToC. Thus, > there is net new content which needs to be written down - particularly > around the topics of monitoring and debugging. > > -- > sankarshan mukhopadhyay > -- sankarshan mukhopadhyay From sankarshan.mukhopadhyay at gmail.com Thu May 25 16:03:51 2017 From: sankarshan.mukhopadhyay at gmail.com (Sankarshan Mukhopadhyay) Date: Thu, 25 May 2017 21:33:51 +0530 Subject: [Tendrl-devel] Ansible installation methods and playbooks for Tendrl In-Reply-To: References: Message-ID: On Tue, May 23, 2017 at 8:48 PM, Mrugesh Karnik wrote: > On 23 May 2017 at 08:11, Sankarshan Mukhopadhyay > wrote: >> + the playbooks will be made available through the tendrl-ansible >> repository, preferably as a new branch >> >> + once these are available; the developers would be testing the >> deployment with these against the current builds >> >> + the playbooks will continue to receive contributions from all Tendrl >> contributors and would thus form the default installation method >> recommended by Tendrl >> >> Please append/amend as appropriate. > > I'm looking through the various deployment related sanity checks and > the ansible playbooks this week. I'll look at what gaps exist between > the two sets of playbooks. I currently believe that the branch drawn > up on tendrl-ansible based on the usmqe playbooks should represent > only the gaps filled in, rather than a wholesale update. In any case, > if the usmqe playbooks are a substantial update, we can look at > replacing the tendrl-ansible playbooks with them. > I agree with the need to do due diligence. From what I know - the playbooks used for building the sandboxes *just work* - can we actually make them part of the tendrl-ansible repo as well? Perhaps as contribs - since Nigel is the author! If the playbooks from usmqe are to be the only way to deploy Tendrl via ansible - they'd need more testing before we make them as such. -- sankarshan mukhopadhyay From japplewh at redhat.com Thu May 25 18:05:38 2017 From: japplewh at redhat.com (Jeff Applewhite) Date: Thu, 25 May 2017 18:05:38 +0000 Subject: [Tendrl-devel] Ansible installation methods and playbooks for Tendrl In-Reply-To: References: Message-ID: Overall I think it just works and we should move ahead. There are a couple of issues, none of which are serious, but I'd like to address them after the migration. On Thu, May 25, 2017 at 12:06 PM Sankarshan Mukhopadhyay < sankarshan.mukhopadhyay at gmail.com> wrote: > On Tue, May 23, 2017 at 8:48 PM, Mrugesh Karnik > wrote: > > On 23 May 2017 at 08:11, Sankarshan Mukhopadhyay > > wrote: > >> + the playbooks will be made available through the tendrl-ansible > >> repository, preferably as a new branch > >> > >> + once these are available; the developers would be testing the > >> deployment with these against the current builds > >> > >> + the playbooks will continue to receive contributions from all Tendrl > >> contributors and would thus form the default installation method > >> recommended by Tendrl > >> > >> Please append/amend as appropriate. > > > > I'm looking through the various deployment related sanity checks and > > the ansible playbooks this week. I'll look at what gaps exist between > > the two sets of playbooks. I currently believe that the branch drawn > > up on tendrl-ansible based on the usmqe playbooks should represent > > only the gaps filled in, rather than a wholesale update. In any case, > > if the usmqe playbooks are a substantial update, we can look at > > replacing the tendrl-ansible playbooks with them. > > > > I agree with the need to do due diligence. From what I know - the > playbooks used for building the sandboxes *just work* - can we > actually make them part of the tendrl-ansible repo as well? Perhaps as > contribs - since Nigel is the author! > > If the playbooks from usmqe are to be the only way to deploy Tendrl > via ansible - they'd need more testing before we make them as such. > > -- > sankarshan mukhopadhyay > > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > -- Jeff Applewhite Principal Product Manager From mbukatov at redhat.com Thu May 25 18:10:09 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Thu, 25 May 2017 20:10:09 +0200 Subject: [Tendrl-devel] graphite restart during performance monitoring installation Message-ID: I'm looking at "Performance Monitoring installation"[1] documentation and one step reads: > Restart httpd: systemctl restart httpd To implement that properly in ansible playbook, I need to understand why is this restart required. When I know this, I can make the restart triggered by change which requires the restart, so that the apache will be restarted only when needed. This is useful when one runs the playbook again to check that the machines are configured properly. Thank you for the answer. [1] https://github.com/Tendrl/documentation/wiki/Tendrl-Package-Installation-Reference#performance-monitoring-installation -- Martin Bukatovic USM QE team Red Hat From mbukatov at redhat.com Thu May 25 18:45:09 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Thu, 25 May 2017 20:45:09 +0200 Subject: [Tendrl-devel] Ansible installation methods and playbooks for Tendrl In-Reply-To: References: Message-ID: <3230d93f-8296-efe6-abb4-24c26af63256@redhat.com> On 05/23/2017 05:18 PM, Mrugesh Karnik wrote: > On 23 May 2017 at 08:11, Sankarshan Mukhopadhyay > wrote: >> + the playbooks will be made available through the tendrl-ansible >> repository, preferably as a new branch >> >> + once these are available; the developers would be testing the >> deployment with these against the current builds >> >> + the playbooks will continue to receive contributions from all Tendrl >> contributors and would thus form the default installation method >> recommended by Tendrl >> >> Please append/amend as appropriate. > > I'm looking through the various deployment related sanity checks and > the ansible playbooks this week. I'll look at what gaps exist between > the two sets of playbooks. I currently believe that the branch drawn > up on tendrl-ansible based on the usmqe playbooks should represent > only the gaps filled in, rather than a wholesale update. In any case, > if the usmqe playbooks are a substantial update, we can look at > replacing the tendrl-ansible playbooks with them. The reason why we are doing this is that both tendrl-ansible and usmqe-setup started with very different approach based on different use cases, but the code idea is the same: to install Tendrl. For this reason, we investigated joining the forces in the past few times already, but we were not able to do it. But now, we finally reached the state when Tendrl is stable (wrt deployment) and documented enough to join forces and make the official ansible code part of the upstream release, so that all people installing tendrl could use it (if they want to - it would not be mandatory - one should be still able to install tendrl without it just by following documentation). For this reason, it needs to be a big change: there are many details in current tendrl-ansible which are not acceptable for general official playbook (hardcoded ulrs for both upstream and downstream packages for example), while on the other hand, qe playbooks have wrong structure and contains many features unsuitable for upstream official playbooks as well ... My idea is to create 3 roles, from the code qe team created for usmqe-setup, based on rpm installation doc[1] and check what is different in current tendrl-ansible: * tendrl-server * tendrl-performance-monitoring * tendrl-node These roles would automate configuration and setup described in the linked document with the following expectations: * repositories required for the role to work (with packages being installed) are already enabled * no changes done for firewall or selinux * the role should be reusable both upstream and downstream without any changes * these roles should be high quality and usable in any use case (real deployment, testing, vagrant setup for poc ...) QE team plans to reuse these roles for setup of test enviroment both downstream and upstream. Besides the 3 main roles (matching roles described in docs[1]), there will be role for enabling upstream corp repo, and playbook(s) (or roles if needed) for other things like firewall/selinux setup, checks proposed by jeff and so on. Then there will be site.yml.example file combining this together (as it's done in ceph-ansible) - which means that the admin would need to tweak the playbook a bit based on his needs, and some vagrant related code. Since this will be a general ansible repo, vagrant will not be promoted as a main use case as it's done in current tendrl-ansible, but it will still work. Does this make sense for you? [1] https://github.com/Tendrl/documentation/wiki/Tendrl-Package-Installation-Reference#performance-monitoring-installation -- Martin Bukatovic USM QE team Red Hat From mbukatov at redhat.com Fri May 26 16:38:13 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Fri, 26 May 2017 18:38:13 +0200 Subject: [Tendrl-devel] ceph-installer on tendrl server Message-ID: Dear all, I see that in "Tendrl Package Installation Reference"[1] document, there is a step to install ceph-installer. Is this step mandatory? I mean, are we saying that one have to install ceph-installer on the server no matter if tendrl will be used to manage ceph? I just want to make this clear, so that we can write this down into the docs to prevent confusion and design ansible roles accordingly. [1] https://github.com/Tendrl/documentation/wiki/Tendrl-Package-Installation-Reference#performance-monitoring-installation -- Martin Bukatovic USM QE team Red Hat From rkanade at redhat.com Fri May 26 16:59:30 2017 From: rkanade at redhat.com (Rohan Kanade) Date: Fri, 26 May 2017 22:29:30 +0530 Subject: [Tendrl-devel] ceph-installer on tendrl server In-Reply-To: References: Message-ID: Thanks for pointing this out, Fixed, and no we dont want users to install ceph-installer unless they want to manage ceph On Fri, May 26, 2017 at 10:08 PM, Martin Bukatovic wrote: > Dear all, > > I see that in "Tendrl Package Installation Reference"[1] document, > there is a step to install ceph-installer. > > Is this step mandatory? I mean, are we saying that one have to > install ceph-installer on the server no matter if tendrl will > be used to manage ceph? I just want to make this clear, so that > we can write this down into the docs to prevent confusion and > design ansible roles accordingly. > > [1] > https://github.com/Tendrl/documentation/wiki/Tendrl-Package-Installation- > Reference#performance-monitoring-installation > > -- > Martin Bukatovic > USM QE team > Red Hat > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From nthomas at redhat.com Fri May 26 17:18:06 2017 From: nthomas at redhat.com (Nishanth Thomas) Date: Fri, 26 May 2017 22:48:06 +0530 Subject: [Tendrl-devel] tendrl-release v1.3.0 (beta2) is available Message-ID: Hello, The Tendrl team is happy to present release v1.3.0 beta2, with the list of issues fixed in this build. - tendrl-commons v1.3.0 https://github.com/Tendrl/commons/issues/486 - tendrl-node-agent v1.3.0 https://github.com/Tendrl/node-agent/issues/476 https://github.com/Tendrl/node-agent/issues/469 https://github.com/Tendrl/node-agent/issues/452 https://github.com/Tendrl/node-agent/issues/462 https://github.com/Tendrl/node-agent/issues/458 https://github.com/Tendrl/node-agent/issues/454 https://github.com/Tendrl/node-agent/issues/453 https://github.com/Tendrl/node-agent/issues/435 https://github.com/Tendrl/node-agent/issues/451 - tendrl-ceph-integration v1.3.0: https://github.com/Tendrl/ceph-integration/issues/246 https://github.com/Tendrl/ceph-integration/issues/245 - tendrl-node-monitoring v1.3.0 https://github.com/Tendrl/node-monitoring/issues/20 - tendrl-api v1.3.0 https://github.com/Tendrl/api/issues/180 https://github.com/Tendrl/api/issues/179 https://github.com/Tendrl/api/issues/173 - tendrl-dashboard v1.3.0 https://github.com/Tendrl/dashboard/issues/361 https://github.com/Tendrl/dashboard/issues/358 https://github.com/Tendrl/dashboard/issues/353 https://github.com/Tendrl/dashboard/issues/375 https://github.com/Tendrl/dashboard/issues/354 https://github.com/Tendrl/dashboard/issues/342 Other components release: - tendrl-alerting v1.3.0 - tendrl-gluster-integration v1.3.0 - tendrl-performance-monitoring v1.3.0 Known open issues: - https://github.com/Tendrl/api/issues/183 Release wiki: https://github.com/Tendrl/documentation/wiki/Tendrl-release- v1.3.0B Cheers! From mbukatov at redhat.com Mon May 29 14:26:36 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Mon, 29 May 2017 16:26:36 +0200 Subject: [Tendrl-devel] installation of tendrl-alerting Message-ID: Dear all, working on cleanup of ansible roles for usptream[3], I have another question: who is responsible for installation of tendrl-alerting? Is it tendrl node agent or admin (so that it would be part of setup automated by tendrl-ansible roles)? The answer is not provided by Tendrl Package Installation Reference document[1], neither it's clear from the issue related to this question[2] (still open). Moreover, this is related to question of mine I have not yet get a good answer: Is the following assumption correct? On Storage None, one have to install and configure tendrl-node-agent and node monitoring - the rest is handled by node agent. Could we check that the status here is and add the description into the installation document[1]? Thank you. [1] https://github.com/Tendrl/documentation/wiki/Tendrl-Package-Installation-Reference#performance-monitoring-installation [2] https://github.com/Tendrl/documentation/issues/79 [3] https://tendrl.atlassian.net/browse/TEN-257 -- Martin Bukatovic USM QE team Red Hat From anbabu at redhat.com Mon May 29 15:25:26 2017 From: anbabu at redhat.com (Anmol Babu) Date: Mon, 29 May 2017 11:25:26 -0400 (EDT) Subject: [Tendrl-devel] installation of tendrl-alerting In-Reply-To: References: Message-ID: <721210010.12866918.1496071526248.JavaMail.zimbra@redhat.com> Martin, tendrl-alerting is a module that needs to be installed explicitly by a tendrl-user and this can be done on any of the nodes(either any storage node or on the etcd/tendrl ui/api node) Note: Currently tendrl-alerting is not highly available and at any given point in time, it can be running on only any one of the nodes in accordance with the above... The install instructions for the alerting module can be found @ https://github.com/Tendrl/alerting/blob/develop/doc/source/installation.rst You are right these steps need to be updated to the Tendrl package installation reference wiki but I seem not to have permissions to either edit or even send a PR on the wiki... @nthomas-redhat @r0h4n @shtripat could any of you please do this. It essentially only needs to be a copy paste of contents of https://github.com/Tendrl/alerting/blob/develop/doc/source/installation.rst to the Tendrl package installation reference wiki... Anmol Babu Software Engineer Red Hat India Pvt Ltd M: 8884245644 ----- Original Message ----- From: "Martin Bukatovic" To: tendrl-devel at redhat.com Sent: Monday, May 29, 2017 7:56:36 PM Subject: [Tendrl-devel] installation of tendrl-alerting Dear all, working on cleanup of ansible roles for usptream[3], I have another question: who is responsible for installation of tendrl-alerting? Is it tendrl node agent or admin (so that it would be part of setup automated by tendrl-ansible roles)? The answer is not provided by Tendrl Package Installation Reference document[1], neither it's clear from the issue related to this question[2] (still open). Moreover, this is related to question of mine I have not yet get a good answer: Is the following assumption correct? On Storage None, one have to install and configure tendrl-node-agent and node monitoring - the rest is handled by node agent. Could we check that the status here is and add the description into the installation document[1]? Thank you. [1] https://github.com/Tendrl/documentation/wiki/Tendrl-Package-Installation-Reference#performance-monitoring-installation [2] https://github.com/Tendrl/documentation/issues/79 [3] https://tendrl.atlassian.net/browse/TEN-257 -- Martin Bukatovic USM QE team Red Hat _______________________________________________ Tendrl-devel mailing list Tendrl-devel at redhat.com https://www.redhat.com/mailman/listinfo/tendrl-devel From sankarshan at redhat.com Mon May 29 15:30:36 2017 From: sankarshan at redhat.com (sankarshan) Date: Mon, 29 May 2017 21:00:36 +0530 Subject: [Tendrl-devel] installation of tendrl-alerting In-Reply-To: <721210010.12866918.1496071526248.JavaMail.zimbra@redhat.com> References: <721210010.12866918.1496071526248.JavaMail.zimbra@redhat.com> Message-ID: On 29 May 2017 at 20:55, Anmol Babu wrote: > You are right these steps need to be updated to the Tendrl package installation reference wiki but I seem not to have permissions to either edit or even send a PR on the wiki... > @nthomas-redhat @r0h4n @shtripat could any of you please do this. It essentially only needs to be a copy paste of contents of https://github.com/Tendrl/alerting/blob/develop/doc/source/installation.rst > to the Tendrl package installation reference wiki... Fixed the perms. Should be working for you. From anbabu at redhat.com Mon May 29 15:45:31 2017 From: anbabu at redhat.com (Anmol Babu) Date: Mon, 29 May 2017 11:45:31 -0400 (EDT) Subject: [Tendrl-devel] installation of tendrl-alerting In-Reply-To: References: <721210010.12866918.1496071526248.JavaMail.zimbra@redhat.com> Message-ID: <2124589525.12869336.1496072731628.JavaMail.zimbra@redhat.com> Thanks Sankarshan for the permissions... Updated the doc now... Anmol Babu Software Engineer Red Hat India Pvt Ltd M: 8884245644 ----- Original Message ----- From: "sankarshan" To: "Mailing list for the contributors to the Tendrl project" Sent: Monday, May 29, 2017 9:00:36 PM Subject: Re: [Tendrl-devel] installation of tendrl-alerting On 29 May 2017 at 20:55, Anmol Babu wrote: > You are right these steps need to be updated to the Tendrl package installation reference wiki but I seem not to have permissions to either edit or even send a PR on the wiki... > @nthomas-redhat @r0h4n @shtripat could any of you please do this. It essentially only needs to be a copy paste of contents of https://github.com/Tendrl/alerting/blob/develop/doc/source/installation.rst > to the Tendrl package installation reference wiki... Fixed the perms. Should be working for you. _______________________________________________ Tendrl-devel mailing list Tendrl-devel at redhat.com https://www.redhat.com/mailman/listinfo/tendrl-devel From mbukatov at redhat.com Mon May 29 16:08:31 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Mon, 29 May 2017 18:08:31 +0200 Subject: [Tendrl-devel] installation of tendrl-alerting In-Reply-To: References: <721210010.12866918.1496071526248.JavaMail.zimbra@redhat.com> Message-ID: On 05/29/2017 05:30 PM, sankarshan wrote: > On 29 May 2017 at 20:55, Anmol Babu wrote: >> You are right these steps need to be updated to the Tendrl package installation reference wiki but I seem not to have permissions to either edit or even send a PR on the wiki... >> @nthomas-redhat @r0h4n @shtripat could any of you please do this. It essentially only needs to be a copy paste of contents of https://github.com/Tendrl/alerting/blob/develop/doc/source/installation.rst >> to the Tendrl package installation reference wiki... > > Fixed the perms. Should be working for you. I see the update on the wiki and I understand it that there is upstream agreement about how this component should work. I'm going to take this into account when working on tend ansible cleanup/merge[1] - it looks like there will be another role for the alerting component Filip or I will recheck the issue[2] tomorrow. [1] https://tendrl.atlassian.net/browse/TEN-257 [2] https://github.com/Tendrl/documentation/issues/79 Thank you all. -- Martin Bukatovic USM QE team Red Hat From mbukatov at redhat.com Tue May 30 14:18:14 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Tue, 30 May 2017 16:18:14 +0200 Subject: [Tendrl-devel] extraneous packages in tendrl dependencies repository In-Reply-To: References: <6c43e4c6-6ee9-ee78-9812-c3d629ec69cb@redhat.com> Message-ID: On 05/22/2017 02:37 PM, Nishanth Thomas wrote: > Thanks for pointing this out > Earlier these packages were not available in epel - this got added > recently. There is specifically no reason to pin down to that old version > and we are making the changes to fix it. That's good to hear! This means that we can drop the packages from the repository ... so could we do that? I still see them there, which makes rmdeplint tests fail: https://ci.centos.org/view/Tendrl/job/tendrl-0-2-package-validation-rpmdeplint/347/testReport/ Thank you. -- Martin Bukatovic USM QE team Red Hat From mkudlej at redhat.com Tue May 30 14:23:20 2017 From: mkudlej at redhat.com (Martin Kudlej) Date: Tue, 30 May 2017 16:23:20 +0200 Subject: [Tendrl-devel] REST API consistency In-Reply-To: <8e1e89a1-2106-0209-a13b-96186d75fb60@redhat.com> References: <8e1e89a1-2106-0209-a13b-96186d75fb60@redhat.com> Message-ID: Hi all, I would like to ask if there is any movement in this area? On 04/26/2017 09:58 AM, Martin Kudlej wrote: > Hi all, > > I think there is still time to unify Tendrl API functions according some rules. > I've filed https://github.com/Tendrl/documentation/issues/70 couple of months ago without response. > > One example for all. > There are these functions for Ceph cluster(Flows on cluster): > "name": "CephCreatePool", > "method": "POST", > > "name": "CephCreateECProfile", > "method": "POST", > > "name": "CephDeleteECProfile", > "method": "DELETE", > > "name": "CephCreateRbd", > "method": "POST", > > "name": "CephDeleteRbd", > "method": "DELETE", > > "name": "CephResizeRbd", > "method": "PUT", > > "name": "CephDeletePool", > "method": "DELETE", > > "name": "CephUpdatePool", > "method": "PUT", > > and according documentation there should be also > "name": "CephPoolList" > "method": "GET" > > I think that there should be also "GET" "CephPool" because of performance. > > I think that these functions should be transformed avoiding verb duplicity. Also I believe if API > function names are renamed integration with other projects will be much easier. > > Pool CRUD > "name": ":ceph_cluster_id:/pool", > "method": "POST", > > "name": ":ceph_cluster_id:/pool/:ceph_pool_id:", > "method": "GET", > > "name": ":ceph_cluster_id:/pool/:ceph_pool_id:", > "method": "PUT", > > "name": ":ceph_cluster_id:/pool/:ceph_pool_id:", > "method": "DELETE", > > RBD CRUD > > "name": ":ceph_cluster_id:/rbd", > "method": "POST", > > "name": ":ceph_cluster_id:/rbd/:ceph_pool_id:", > "method": "GET", > > "name": ":ceph_cluster_id:/rbd/:ceph_pool_id:", > "method": "PUT", > > "name": ":ceph_cluster_id:/rbd/:ceph_pool_id:", > "method": "DELETE", > > EC Profile CRUD > > "name": ":ceph_cluster_id:/ecprofile", > "method": "POST", > > "name": ":ceph_cluster_id:/ecprofile/:ec_profile_id:", > "method": "GET", > > (update if it is possible) > "name": ":ceph_cluster_id:/ecprofile/:ec_profile_id:", > "method": "PUT", > > "name": ":ceph_cluster_id:/ecprofile/:ec_profile_id:", > "method": "DELETE", > > > Lists > > "name": ":ceph_cluster_id:/pool" > "method": "GET" > > "name": ":ceph_cluster_id:/rbd" > "method": "GET" > > "name": ":ceph_cluster_id:/ecprofile" > "method": "GET" > > Please consider this rename until is not too late and there are not many projects which are > integrated with Tendrl. > -- Best Regards, Martin Kudlej. RHSC/USM Senior Quality Assurance Engineer Red Hat Czech s.r.o. Phone: +420 532 294 155 E-mail:mkudlej at redhat.com IRC: mkudlej at #brno, #gluster, #storage-qa, #rhs, #rh-ceph, #usm-meeting @ redhat #tendrl-devel @ freenode From fbalak at redhat.com Tue May 30 15:12:46 2017 From: fbalak at redhat.com (Filip Balak) Date: Tue, 30 May 2017 11:12:46 -0400 (EDT) Subject: [Tendrl-devel] node-agent/node-monitoring behaviour In-Reply-To: <42468646.14149244.1496156537095.JavaMail.zimbra@redhat.com> Message-ID: <2016149982.14155934.1496157166077.JavaMail.zimbra@redhat.com> Hi Anmol, is node-monitoring somehow managed by node-agent? For example does node-agent start and stop node-monitoring service? According to this comment[1] it seems that node-agent stops the service but it doesn't start the service again. Would it make sense if the node-monitoring service was started by node-agent when node-agent is (re)started? Best Regards, Filip Balak [1] https://github.com/Tendrl/dashboard/issues/361#issuecomment-303934353 From anbabu at redhat.com Tue May 30 15:31:12 2017 From: anbabu at redhat.com (Anmol Babu) Date: Tue, 30 May 2017 11:31:12 -0400 (EDT) Subject: [Tendrl-devel] node-agent/node-monitoring behaviour In-Reply-To: <2016149982.14155934.1496157166077.JavaMail.zimbra@redhat.com> References: <2016149982.14155934.1496157166077.JavaMail.zimbra@redhat.com> Message-ID: <2005818679.13199250.1496158272903.JavaMail.zimbra@redhat.com> Hi Filip, The problem is: 1. node-monitoring is an optional module so node-agent managing node-monitoring may not be desirable. 2. node-agent service is required to be running for node-monitoring or any other tendrl service for that matter to be running. Note: reverse is not true. And this is because node-agent is sole means of logging for every tendrl module. Anmol Babu Software Engineer Red Hat India Pvt Ltd M: 8884245644 ----- Original Message ----- From: "Filip Balak" To: "Anmol Babu" Cc: "Mailing list for the contributors to the Tendrl project" Sent: Tuesday, May 30, 2017 8:42:46 PM Subject: node-agent/node-monitoring behaviour Hi Anmol, is node-monitoring somehow managed by node-agent? For example does node-agent start and stop node-monitoring service? According to this comment[1] it seems that node-agent stops the service but it doesn't start the service again. Would it make sense if the node-monitoring service was started by node-agent when node-agent is (re)started? Best Regards, Filip Balak [1] https://github.com/Tendrl/dashboard/issues/361#issuecomment-303934353 From dahorak at redhat.com Wed May 31 08:14:05 2017 From: dahorak at redhat.com (=?UTF-8?Q?Daniel_Hor=c3=a1k?=) Date: Wed, 31 May 2017 10:14:05 +0200 Subject: [Tendrl-devel] File Share (aka Gluster Volume) creation wizard Message-ID: Hi Ju, I have few questions related to File Share (aka Gluster Volume) creation wizard. What exactly means the "Bricks per Replica Set" value on Create File Share - Layout[1] page? Is it related to replication or distribution? I think it should be related to "distribution" (because Replication might be set only to 2 or 3), but I'm not sure how exactly. And also there seems to be some misunderstanding between the design and real implementation[2]. [1] https://redhat.invisionapp.com/share/Q78YMAVDJ#/screens/228991297 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1456418#c3 Thanks, Daniel From mbukatov at redhat.com Wed May 31 08:25:05 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Wed, 31 May 2017 10:25:05 +0200 Subject: [Tendrl-devel] installation of monitoring and alerting Message-ID: <28595887-4cbf-1cb9-cb60-2778e94a3733@redhat.com> Dear all, I have another questiong about Tendrl installation. In "Tendrl Package Installation Reference" document[1], both *Performance Moritoring* and *Alerting* roles states that one can install mentioned components on different machines: > Performance Monitoring can be installed either on: > * Node where etcd is installed > * New node > > Alerting can be installed either on: > > * Node where etcd is installed > * Storage node I would like to get this clarified a bit. 1) What are pros and cons of each installation configuration? Based on what should one decide on which machine to install the role in both cases? 2) What is suggested safe default? 3) Does "New node" for Performance Monitoring mean a dedicated machine just for this role? 4) Why alternative place for monitoring is "new node", while alerting could be placed on storage node? Is it possible to install monitoring and alerting on single dedicated machine? And would it make sense? Thank you for checking this. -- Martin Bukatovic USM QE team Red Hat From anbabu at redhat.com Wed May 31 10:07:41 2017 From: anbabu at redhat.com (Anmol Babu) Date: Wed, 31 May 2017 06:07:41 -0400 (EDT) Subject: [Tendrl-devel] installation of monitoring and alerting In-Reply-To: <28595887-4cbf-1cb9-cb60-2778e94a3733@redhat.com> References: <28595887-4cbf-1cb9-cb60-2778e94a3733@redhat.com> Message-ID: <1945471781.13697150.1496225261797.JavaMail.zimbra@redhat.com> Hi Martin, As an implementer I have always been trying to keep maximum options open in the code and to suit timelines so that any future changes are easy to adapt to. And doing so, I totally missed this in the midst of deliverable s and timelines.. Thanks a lot for bringing up this discussion here... Important points to note before diving into the questions: 1. The performance-monitoring installation brings in(installs) and configures graphite. 2. All nodes push stats to the graphite installed by performance-monitoring application.. 3. Writes to graphite induce read/write disk operations as the stats are maintained in files by whisper database(part of graphite stack). Note: Metrics get fed into the graphite stack via the Carbon service, which writes data out to Whisper databases for long-term storage. Now the questions: >1) What are pros and cons of each installation configuration? Based on > what should one decide on which machine to install the role in both > cases? I think the answer to this is mainly governed by: a. Points 1 to 3 above under "Important points to note..." b. performance-monitoring application does lots of etcd interactions and also has interactions from tendrl/api which makes me say that having them co-resident would be beneficial from network utilization perspective may not be a major gain though... c. To avoid having to dedicate a node for this purpose one can as well use the same node for installing etcd, api, dashboard, performance-monitoring and alerting application along with their dependent service node-agent. d. Point c in essence also suggests that this can be essentially deployed on a storage node as well. Note: Having etcd and api along with performance-monitoring, node-monitoring and alerting might bring in resource utilization related issues. But this might need to be confirmed via tests... >2) What is suggested safe default? The answer to this varies in accordance with considerations above... >3) Does "New node" for Performance Monitoring mean a dedicated machine > just for this role? Yes, based on considerations above, if this is inevitable, there is nothing in code that stops such a deployment... >4) Why alternative place for monitoring is "new node", while alerting > could be placed on storage node? Is it possible to install monitoring > and alerting on single dedicated machine? And would it make sense? The alternatives to both applications from the perspective of code are the same.. Its just a fix required on documentation if it suggests otherwise.. Having performance-monitoring and alerting on a dedicated machine might make sense but yes as said above it all depends on considerations above... To summarise, I would like to say that definitive and/or quantitative answers to most of these questions, can be made after: 1. We test each of these deployment scenarios 2. We perform scale tests to see how tendrl scales... @Nishanth, @shubhendu, @rohan and @Mrugesh Please provide your valuable thoughts... Anmol Babu Software Engineer Red Hat India Pvt Ltd M: 8884245644 ----- Original Message ----- From: "Martin Bukatovic" To: tendrl-devel at redhat.com Sent: Wednesday, May 31, 2017 1:55:05 PM Subject: [Tendrl-devel] installation of monitoring and alerting Dear all, I have another questiong about Tendrl installation. In "Tendrl Package Installation Reference" document[1], both *Performance Moritoring* and *Alerting* roles states that one can install mentioned components on different machines: > Performance Monitoring can be installed either on: > * Node where etcd is installed > * New node > > Alerting can be installed either on: > > * Node where etcd is installed > * Storage node I would like to get this clarified a bit. 1) What are pros and cons of each installation configuration? Based on what should one decide on which machine to install the role in both cases? 2) What is suggested safe default? 3) Does "New node" for Performance Monitoring mean a dedicated machine just for this role? 4) Why alternative place for monitoring is "new node", while alerting could be placed on storage node? Is it possible to install monitoring and alerting on single dedicated machine? And would it make sense? Thank you for checking this. -- Martin Bukatovic USM QE team Red Hat _______________________________________________ Tendrl-devel mailing list Tendrl-devel at redhat.com https://www.redhat.com/mailman/listinfo/tendrl-devel From anbabu at redhat.com Wed May 31 10:20:08 2017 From: anbabu at redhat.com (Anmol Babu) Date: Wed, 31 May 2017 06:20:08 -0400 (EDT) Subject: [Tendrl-devel] installation of monitoring and alerting In-Reply-To: <1945471781.13697150.1496225261797.JavaMail.zimbra@redhat.com> References: <28595887-4cbf-1cb9-cb60-2778e94a3733@redhat.com> <1945471781.13697150.1496225261797.JavaMail.zimbra@redhat.com> Message-ID: <2103355614.13700197.1496226008656.JavaMail.zimbra@redhat.com> Please also note, if we decide to put performance-monitoring and graphite on different nodes(in case graphite is already deployed and is in use even before tendrl installation), I would like to say that this can be achieved with no to very less(might require some spec changes but needs to be verified thoroughly though) changes in performance-monitoring code-base.. Anmol Babu Software Engineer Red Hat India Pvt Ltd M: 8884245644 ----- Original Message ----- From: "Anmol Babu" To: "Mailing list for the contributors to the Tendrl project" Sent: Wednesday, May 31, 2017 3:37:41 PM Subject: Re: [Tendrl-devel] installation of monitoring and alerting Hi Martin, As an implementer I have always been trying to keep maximum options open in the code and to suit timelines so that any future changes are easy to adapt to. And doing so, I totally missed this in the midst of deliverable s and timelines.. Thanks a lot for bringing up this discussion here... Important points to note before diving into the questions: 1. The performance-monitoring installation brings in(installs) and configures graphite. 2. All nodes push stats to the graphite installed by performance-monitoring application.. 3. Writes to graphite induce read/write disk operations as the stats are maintained in files by whisper database(part of graphite stack). Note: Metrics get fed into the graphite stack via the Carbon service, which writes data out to Whisper databases for long-term storage. Now the questions: >1) What are pros and cons of each installation configuration? Based on > what should one decide on which machine to install the role in both > cases? I think the answer to this is mainly governed by: a. Points 1 to 3 above under "Important points to note..." b. performance-monitoring application does lots of etcd interactions and also has interactions from tendrl/api which makes me say that having them co-resident would be beneficial from network utilization perspective may not be a major gain though... c. To avoid having to dedicate a node for this purpose one can as well use the same node for installing etcd, api, dashboard, performance-monitoring and alerting application along with their dependent service node-agent. d. Point c in essence also suggests that this can be essentially deployed on a storage node as well. Note: Having etcd and api along with performance-monitoring, node-monitoring and alerting might bring in resource utilization related issues. But this might need to be confirmed via tests... >2) What is suggested safe default? The answer to this varies in accordance with considerations above... >3) Does "New node" for Performance Monitoring mean a dedicated machine > just for this role? Yes, based on considerations above, if this is inevitable, there is nothing in code that stops such a deployment... >4) Why alternative place for monitoring is "new node", while alerting > could be placed on storage node? Is it possible to install monitoring > and alerting on single dedicated machine? And would it make sense? The alternatives to both applications from the perspective of code are the same.. Its just a fix required on documentation if it suggests otherwise.. Having performance-monitoring and alerting on a dedicated machine might make sense but yes as said above it all depends on considerations above... To summarise, I would like to say that definitive and/or quantitative answers to most of these questions, can be made after: 1. We test each of these deployment scenarios 2. We perform scale tests to see how tendrl scales... @Nishanth, @shubhendu, @rohan and @Mrugesh Please provide your valuable thoughts... Anmol Babu Software Engineer Red Hat India Pvt Ltd M: 8884245644 ----- Original Message ----- From: "Martin Bukatovic" To: tendrl-devel at redhat.com Sent: Wednesday, May 31, 2017 1:55:05 PM Subject: [Tendrl-devel] installation of monitoring and alerting Dear all, I have another questiong about Tendrl installation. In "Tendrl Package Installation Reference" document[1], both *Performance Moritoring* and *Alerting* roles states that one can install mentioned components on different machines: > Performance Monitoring can be installed either on: > * Node where etcd is installed > * New node > > Alerting can be installed either on: > > * Node where etcd is installed > * Storage node I would like to get this clarified a bit. 1) What are pros and cons of each installation configuration? Based on what should one decide on which machine to install the role in both cases? 2) What is suggested safe default? 3) Does "New node" for Performance Monitoring mean a dedicated machine just for this role? 4) Why alternative place for monitoring is "new node", while alerting could be placed on storage node? Is it possible to install monitoring and alerting on single dedicated machine? And would it make sense? Thank you for checking this. -- Martin Bukatovic USM QE team Red Hat _______________________________________________ Tendrl-devel mailing list Tendrl-devel at redhat.com https://www.redhat.com/mailman/listinfo/tendrl-devel From japplewh at redhat.com Wed May 31 13:26:43 2017 From: japplewh at redhat.com (Jeff Applewhite) Date: Wed, 31 May 2017 09:26:43 -0400 Subject: [Tendrl-devel] installation of monitoring and alerting In-Reply-To: <2103355614.13700197.1496226008656.JavaMail.zimbra@redhat.com> References: <28595887-4cbf-1cb9-cb60-2778e94a3733@redhat.com> <1945471781.13697150.1496225261797.JavaMail.zimbra@redhat.com> <2103355614.13700197.1496226008656.JavaMail.zimbra@redhat.com> Message-ID: For now why don't we keep the installation documentation and pathway simple. This would mean a a single Tendrl node including all required components with a *manual* (no Ansible) process to setup HA if the admin so desires. This would involved replicating etcd to two (or more) other nodes primarily (since the failover process has also not been tested yet). This is simply so that in the case of primary node failure a recovery could be achieved based on a replica etcd node. No need to setup API on the secondary at this point. That can come later as we prove out those scenarios. Let's also put a *note* in the documentation that running performance monitoring on another node is theoretically possible but not tested. On Wed, May 31, 2017 at 6:20 AM, Anmol Babu wrote: > Please also note, > if we decide to put performance-monitoring and graphite on different > nodes(in case graphite is already deployed and is in use even before tendrl > installation), > I would like to say that this can be achieved with no to very less(might > require some spec changes but needs to be verified thoroughly though) > changes in performance-monitoring code-base.. > > Anmol Babu > Software Engineer > Red Hat India Pvt Ltd > M: 8884245644 > > ----- Original Message ----- > From: "Anmol Babu" > To: "Mailing list for the contributors to the Tendrl project" < > tendrl-devel at redhat.com> > Sent: Wednesday, May 31, 2017 3:37:41 PM > Subject: Re: [Tendrl-devel] installation of monitoring and alerting > > Hi Martin, > > As an implementer I have always been trying to keep maximum options open > in the code and to suit timelines so that any future changes are easy to > adapt to. > And doing so, I totally missed this in the midst of deliverable s and > timelines.. > Thanks a lot for bringing up this discussion here... > > Important points to note before diving into the questions: > > 1. The performance-monitoring installation brings in(installs) and > configures graphite. > 2. All nodes push stats to the graphite installed by > performance-monitoring application.. > 3. Writes to graphite induce read/write disk operations as the stats are > maintained in files by whisper database(part of graphite stack). > Note: Metrics get fed into the graphite stack via the Carbon service, > which writes data out to Whisper databases for long-term storage. > > Now the questions: > > >1) What are pros and cons of each installation configuration? Based on > > what should one decide on which machine to install the role in both > > cases? > > I think the answer to this is mainly governed by: > a. Points 1 to 3 above under "Important points to note..." > b. performance-monitoring application does lots of etcd interactions and > also has interactions from tendrl/api which makes me say that having them > co-resident > would be beneficial from network utilization perspective may not be a > major gain though... > c. To avoid having to dedicate a node for this purpose one can as well use > the same node for installing etcd, api, dashboard, performance-monitoring > and alerting application along with their dependent service node-agent. > d. Point c in essence also suggests that this can be essentially deployed > on a storage node as well. > Note: Having etcd and api along with performance-monitoring, > node-monitoring and alerting might bring in resource utilization related > issues. > But this might need to be confirmed via tests... > > >2) What is suggested safe default? > > The answer to this varies in accordance with considerations above... > > >3) Does "New node" for Performance Monitoring mean a dedicated machine > > just for this role? > > Yes, based on considerations above, if this is inevitable, there is > nothing in code that stops such a deployment... > > >4) Why alternative place for monitoring is "new node", while alerting > > could be placed on storage node? Is it possible to install monitoring > > and alerting on single dedicated machine? And would it make sense? > > The alternatives to both applications from the perspective of code are the > same.. > Its just a fix required on documentation if it suggests otherwise.. > Having performance-monitoring and alerting on a dedicated machine might > make sense but yes as said above it all depends on considerations above... > > To summarise, I would like to say that definitive and/or quantitative > answers to most of these questions, can be made after: > > 1. We test each of these deployment scenarios > 2. We perform scale tests to see how tendrl scales... > > > @Nishanth, @shubhendu, @rohan and @Mrugesh Please provide your valuable > thoughts... > > > Anmol Babu > Software Engineer > Red Hat India Pvt Ltd > M: 8884245644 > > ----- Original Message ----- > From: "Martin Bukatovic" > To: tendrl-devel at redhat.com > Sent: Wednesday, May 31, 2017 1:55:05 PM > Subject: [Tendrl-devel] installation of monitoring and alerting > > Dear all, > > I have another questiong about Tendrl installation. In "Tendrl Package > Installation Reference" document[1], both *Performance Moritoring* and > *Alerting* roles states that one can install mentioned components on > different machines: > > > Performance Monitoring can be installed either on: > > * Node where etcd is installed > > * New node > > > > Alerting can be installed either on: > > > > * Node where etcd is installed > > * Storage node > > I would like to get this clarified a bit. > > 1) What are pros and cons of each installation configuration? Based on > what should one decide on which machine to install the role in both > cases? > 2) What is suggested safe default? > 3) Does "New node" for Performance Monitoring mean a dedicated machine > just for this role? > 4) Why alternative place for monitoring is "new node", while alerting > could be placed on storage node? Is it possible to install monitoring > and alerting on single dedicated machine? And would it make sense? > > Thank you for checking this. > > -- > Martin Bukatovic > USM QE team > Red Hat > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > -- Jeff Applewhite Principal Product Manager From mbukatov at redhat.com Wed May 31 13:50:02 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Wed, 31 May 2017 15:50:02 +0200 Subject: [Tendrl-devel] installation of monitoring and alerting In-Reply-To: <1945471781.13697150.1496225261797.JavaMail.zimbra@redhat.com> References: <28595887-4cbf-1cb9-cb60-2778e94a3733@redhat.com> <1945471781.13697150.1496225261797.JavaMail.zimbra@redhat.com> Message-ID: <0c4130c9-4966-0202-ca53-48f687b3ab7d@redhat.com> Hi Anmol, On 05/31/2017 12:07 PM, Anmol Babu wrote: > 1. The performance-monitoring installation brings in(installs) and configures graphite. > 2. All nodes push stats to the graphite installed by performance-monitoring application.. > 3. Writes to graphite induce read/write disk operations as the stats are maintained in files by whisper database(part of graphite stack). > > Note: Metrics get fed into the graphite stack via the Carbon service, which writes data out to Whisper databases for long-term storage. Ok, so performance monitoring machine is a target for performance data pushes from all other Tendrl machines. >> 1) What are pros and cons of each installation configuration? Based on >> what should one decide on which machine to install the role in both >> cases? > > I think the answer to this is mainly governed by: > a. Points 1 to 3 above under "Important points to note..." > b. performance-monitoring application does lots of etcd interactions and also has interactions from tendrl/api which makes me say that having them co-resident > would be beneficial from network utilization perspective may not be a major gain though... > c. To avoid having to dedicate a node for this purpose one can as well use the same node for installing etcd, api, dashboard, performance-monitoring and alerting application along with their dependent service node-agent. > d. Point c in essence also suggests that this can be essentially deployed on a storage node as well. > Note: Having etcd and api along with performance-monitoring, node-monitoring and alerting might bring in resource utilization related issues. > But this might need to be confirmed via tests... And at the same time, performance monitoring communicates a lot with etcd and tendrl api. Ok. So for this reason, it may make sense to deploy it on the Tendrl Server (along with etcd and tendrl api/web). But you are concerned with the resource requirements, especially wrt scaling. But I don't get why it seems a good idea to deploy performance monitoring on storage node. There are even more issues with resource utilization in that case ... >> 2) What is suggested safe default? > > The answer to this varies in accordance with considerations above... I will reply below the summary. >> 3) Does "New node" for Performance Monitoring mean a dedicated machine >> just for this role? > > Yes, based on considerations above, if this is inevitable, there is nothing in code that stops such a deployment... Ok. >> 4) Why alternative place for monitoring is "new node", while alerting >> could be placed on storage node? Is it possible to install monitoring >> and alerting on single dedicated machine? And would it make sense? > > The alternatives to both applications from the perspective of code are the same.. > Its just a fix required on documentation if it suggests otherwise.. > Having performance-monitoring and alerting on a dedicated machine might make sense but yes as said above it all depends on considerations above... You mentioned Performance Monitoring constrains in a clear way, but not for Alerting. I'm interested especially in: * having performance monitoring and alerting on the same machine - * having alerting on storage machine > To summarise, I would like to say that definitive and/or quantitative answers to most of these questions, can be made after: > > 1. We test each of these deployment scenarios > 2. We perform scale tests to see how tendrl scales... So at this point, we don't have enough experiences with the behavior of the system to suggest how to deploy the system exactly. That's fine, but I would suggest to write down a gist of what we know right now directly into the docs. Would it make sense to write something like (here I'm proposing what we could add into the docs and verifying that I understand you at the same time - feel free to correct me and fix/expand the text): For Performance Monitoring: --- suggested update --- This role could be deployed either on Tendrl Server machine (along with tendrl api, etcd, ... as described above) or a dedicated machine. Performance monitoring machine is a target for performance data pushes from all other Tendrl machines and communicates a lot with services on Tendrl Server (etcd and tendrl-api) - which affects network utilization and produces lots of tiny I/O operations. For this reason, it makes sense to deploy it on Tendrl Server, but when you see problems with resource utilization on Tendrl Server, it may be better to go with deployment on a dedicated machine. We have not enough experiences to provide final and exact guidelines here. --- suggested update --- The same should be done for Alerting. For testing, we deploy both alerting and performance monitoring on tendrl server, as can be seen here: https://github.com/Tendrl/usmqe-setup/blob/master/tendrl_server.yml#L18 I understand you reply as a request for us to start tracking resource utilization per services on this type of machine, to get a better data. -- Martin Bukatovic USM QE team Red Hat From mbukatov at redhat.com Wed May 31 16:23:14 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Wed, 31 May 2017 18:23:14 +0200 Subject: [Tendrl-devel] graphite restart during performance monitoring installation In-Reply-To: References: Message-ID: ping On 05/25/2017 08:10 PM, Martin Bukatovic wrote: > I'm looking at "Performance Monitoring installation"[1] > documentation and one step reads: > >> Restart httpd: systemctl restart httpd > > To implement that properly in ansible playbook, I need to > understand why is this restart required. When I know this, > I can make the restart triggered by change which requires > the restart, so that the apache will be restarted only > when needed. This is useful when one runs the playbook > again to check that the machines are configured properly. > > Thank you for the answer. > > [1] > https://github.com/Tendrl/documentation/wiki/Tendrl-Package-Installation-Reference#performance-monitoring-installation -- Martin Bukatovic USM QE team Red Hat From sankarshan at redhat.com Wed May 31 16:34:18 2017 From: sankarshan at redhat.com (sankarshan) Date: Wed, 31 May 2017 22:04:18 +0530 Subject: [Tendrl-devel] graphite restart during performance monitoring installation In-Reply-To: References: Message-ID: Copied Anmol into the thread to take a look at this. On 25 May 2017 at 23:40, Martin Bukatovic wrote: > I'm looking at "Performance Monitoring installation"[1] > documentation and one step reads: > >> Restart httpd: systemctl restart httpd > > To implement that properly in ansible playbook, I need to > understand why is this restart required. When I know this, > I can make the restart triggered by change which requires > the restart, so that the apache will be restarted only > when needed. This is useful when one runs the playbook > again to check that the machines are configured properly. > > Thank you for the answer. > > [1] > https://github.com/Tendrl/documentation/wiki/Tendrl-Package-Installation-Reference#performance-monitoring-installation > > -- > Martin Bukatovic > USM QE team > Red Hat > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From anbabu at redhat.com Wed May 31 17:07:20 2017 From: anbabu at redhat.com (Anmol Babu) Date: Wed, 31 May 2017 13:07:20 -0400 (EDT) Subject: [Tendrl-devel] graphite restart during performance monitoring installation In-Reply-To: References: Message-ID: <1279148099.13836032.1496250440542.JavaMail.zimbra@redhat.com> Hi Martin, We are adding a new httpd configuration as in https://github.com/Tendrl/performance-monitoring/blob/develop/etc/tendrl/performance-monitoring/graphite-web.conf.sample To enable requests on 10080 to be served with graphite-web... This is the step that requires httpd to be restarted after performance-monitoring installation(https://github.com/Tendrl/performance-monitoring/blob/develop/tendrl-performance-monitoring.spec#L47)... Anmol Babu Software Engineer Red Hat India Pvt Ltd M: 8884245644 ----- Original Message ----- From: "sankarshan" To: "Mailing list for the contributors to the Tendrl project" Sent: Wednesday, May 31, 2017 10:04:18 PM Subject: Re: [Tendrl-devel] graphite restart during performance monitoring installation Copied Anmol into the thread to take a look at this. On 25 May 2017 at 23:40, Martin Bukatovic wrote: > I'm looking at "Performance Monitoring installation"[1] > documentation and one step reads: > >> Restart httpd: systemctl restart httpd > > To implement that properly in ansible playbook, I need to > understand why is this restart required. When I know this, > I can make the restart triggered by change which requires > the restart, so that the apache will be restarted only > when needed. This is useful when one runs the playbook > again to check that the machines are configured properly. > > Thank you for the answer. > > [1] > https://github.com/Tendrl/documentation/wiki/Tendrl-Package-Installation-Reference#performance-monitoring-installation > > -- > Martin Bukatovic > USM QE team > Red Hat > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel _______________________________________________ Tendrl-devel mailing list Tendrl-devel at redhat.com https://www.redhat.com/mailman/listinfo/tendrl-devel From mbukatov at redhat.com Wed May 31 17:42:05 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Wed, 31 May 2017 19:42:05 +0200 Subject: [Tendrl-devel] graphite restart during performance monitoring installation In-Reply-To: <1279148099.13836032.1496250440542.JavaMail.zimbra@redhat.com> References: <1279148099.13836032.1496250440542.JavaMail.zimbra@redhat.com> Message-ID: <0aba1a87-5dbd-9fde-a65b-969c24a3f8e2@redhat.com> Hi Anmol, On 05/31/2017 07:07 PM, Anmol Babu wrote: > We are adding a new httpd configuration as in https://github.com/Tendrl/performance-monitoring/blob/develop/etc/tendrl/performance-monitoring/graphite-web.conf.sample > To enable requests on 10080 to be served with graphite-web... > This is the step that requires httpd to be restarted after performance-monitoring installation(https://github.com/Tendrl/performance-monitoring/blob/develop/tendrl-performance-monitoring.spec#L47)... thanks for the reply -- Martin Bukatovic USM QE team Red Hat From mbukatov at redhat.com Wed May 31 19:13:18 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Wed, 31 May 2017 21:13:18 +0200 Subject: [Tendrl-devel] installation of monitoring and alerting In-Reply-To: References: <28595887-4cbf-1cb9-cb60-2778e94a3733@redhat.com> <1945471781.13697150.1496225261797.JavaMail.zimbra@redhat.com> <2103355614.13700197.1496226008656.JavaMail.zimbra@redhat.com> Message-ID: <842b1d20-dcb5-9a82-998f-e6c72d088df3@redhat.com> On 05/31/2017 03:26 PM, Jeff Applewhite wrote: > For now why don't we keep the installation documentation and pathway > simple. > > This would mean a a single Tendrl node including all required components > with a *manual* (no Ansible) process to setup HA if the admin so desires. > This would involved replicating etcd to two (or more) other nodes primarily > (since the failover process has also not been tested yet). This is simply > so that in the case of primary node failure a recovery could be achieved > based on a replica etcd node. No need to setup API on the secondary at this > point. That can come later as we prove out those scenarios. The current documentation is not concerned with HA at all, as we agreed previously to concentrate on the simple setup first. And here I was just trying to clarify setup for this simple deployment scenario as currently described in Installation Reference document. I have good enough understanding of the trade offs here so that I'm not blocked on my work on tendrl-ansible restart. But at the same time, I would like add a reference to the explanation in the docs into the playbook - so that it's clear what can be changed - and for this to happen we need to update the docs a bit. > Let's also put a *note* in the documentation that running performance > monitoring on another node is theoretically possible but not tested. I agree, something like that which would clearly convey the current status as we understand it, I suggested one possible note in another email in this thread. -- Martin Bukatovic USM QE team Red Hat From mbukatov at redhat.com Wed May 31 19:28:41 2017 From: mbukatov at redhat.com (Martin Bukatovic) Date: Wed, 31 May 2017 21:28:41 +0200 Subject: [Tendrl-devel] rename tendrl-api service Message-ID: <43a4a1e3-9124-fcdf-7ea4-02a4e7dd5a59@redhat.com> Could we get this simple rename fix in next upstream build? https://github.com/Tendrl/api/issues/45 I would be nice to have this in asap so that we don't have to change documentation and tendrl-ansible again later. -- Martin Bukatovic USM QE team Red Hat From julim at redhat.com Wed May 31 20:03:00 2017 From: julim at redhat.com (Ju Lim) Date: Wed, 31 May 2017 16:03:00 -0400 Subject: [Tendrl-devel] File Share (aka Gluster Volume) creation wizard In-Reply-To: References: Message-ID: Hi Daniel: Thanks for starting this thread as there appears to be a difference between our design vs. implementation. Some assumptions before I go into answering your questions: For Tendrl, we will initially support 2 types of volumes: - 3-way distributed replicated - Erasure Coded (4+2 with min 6 bricks, 8+3 with min 11 bricks, 8+4 with min 12 bricks) Note: 2-way distributed replicated is left off as we will plan arbiter volumes to cover that scenario in the future. For the 3-way distributed replicated, per the official documentation : The number of bricks must be a multiple of the replica count for a distributed replicated volume. Also, the order in which bricks are specified has a great effect on data protection. Each replica_count consecutive bricks in the list you give will form a replica set, with all replica sets combined into a distribute set. To ensure that replica-set members are not placed on the same node, list the first brick on every server, then the second brick on every server in the same order, and so on. So, in other words, if we had a 3-way dist-rep, that means it?s 3 consecutive bricks to form a replica set. So, if you?re using 6 nodes (with 1 brick each), that would mean 2 replica sets. Ideally, user should not have to input Bricks per Replica set, and System should be able to compute this, i.e. # bricks per replica set == replica_count # replica sets == # of bricks / replica_count Thus, we should remove the ?Bricks per Replica Set? from the design. For EC (distributed dispersed) volumes, the 2 parameters needed are typically: - disperse-data_count == k or the number of bricks that is part of the dispersed volume, excluding the count of the redundant bricks - redundancy_count (optional parameter) == m or # of bricks that can fail without losing data, i.e. m To answer your questions for the Create File Share (aka create gluster volume): What exactly means the "Bricks per Replica Set" value on Create File Share - Layout[1] page? - This was intended to help with the visualization of the layout but I don?t think it?s needed and probably should be removed. (It was a bit of heritage from the old console.) - Is it related to replication or distribution? It isn?t. Some observations from the current implemented Create File Share workflow: - The combined Type (Distributed Replicated) + Replica Count {2 or 3} == 2-way or 3-way distributed replicated. This is not to design as we collapsed this to 3-way distributed replicated for the Type in our design to simplify the user experience. - The Distribution Count is not in our design and it?s not even specified in the Gluster CLI for in the related Tendrl specs that I could find, so I?m not sure what this is for. I thought perhaps it was for something EC-related, but there?s currently no way to specify EC in the UI for creating the volume. Per https://github.com/Tendrl/specifications/blob/33eabe3123b290bd030a94b30fdde709253539f8/specs/segragate_object_specific_flows.adoc, the volume inputs expected are volume.volname, volume.bricks and optionally: - Volume.stripe_count - Volume.replica_count - Volume.arbiter_count - Volume.disperse_count - Volume.disperse_data_count - Volume.redundancy_count - Volume.transport - Volume.force Related References: - Create File Share workflow: https://redhat.invisionapp.com/share/Q78YMAVDJ#/screens/217726640 - Gaps of design vs. implementation: https://bugzilla.redhat.com/show_bug.cgi?id=1456418#c3 - Spec for Create Volume: ? - Tendrl-gdeploy wrapper spec: https://github.com/Tendrl/specifications/blob/8c19e6af7a459bf6a6070fca2bbe3ef710aff819/specs/tendrl_gdeploy_wrapper.adoc - Segregate object specific flows: https://github.com/Tendrl/specifications/blob/33eabe3123b290bd030a94b30fdde709253539f8/specs/segragate_object_specific_flows.adoc - UI code references for distribution count: https://github.com/Tendrl/dashboard/search?utf8=%E2%9C%93&q=distribution+count&type= As a last note, we will update the Create File Share design to remove the 2-way distributed-replicated and the Bricks per Replica Set field, and update the sample workflow. I hope this clarifies any confusion. If needed and it makes sense, please feel free to schedule a meeting to discuss this to get everyone on the same page. Thank you, Ju On Wed, May 31, 2017 at 4:14 AM, Daniel Hor?k wrote: > Hi Ju, > > I have few questions related to File Share (aka Gluster Volume) creation > wizard. > > What exactly means the "Bricks per Replica Set" value on Create File Share > - Layout[1] page? > Is it related to replication or distribution? I think it should be related > to "distribution" (because Replication might be set only to 2 or 3), but > I'm not sure how exactly. > > And also there seems to be some misunderstanding between the design and > real implementation[2]. > > [1] https://redhat.invisionapp.com/share/Q78YMAVDJ#/screens/228991297 > [2] https://bugzilla.redhat.com/show_bug.cgi?id=1456418#c3 > > Thanks, > Daniel >