From mkudlej at redhat.com Tue Nov 1 09:24:36 2016 From: mkudlej at redhat.com (Martin Kudlej) Date: Tue, 1 Nov 2016 10:24:36 +0100 Subject: [Tendrl-devel] CentOS Storage SIG meeting on Friday 28-Oct-2016 at 13:00 UTC In-Reply-To: References: <20161026185118.GD2936@ndevos-x240.usersys.redhat.com> <20161027165114.GG2936@ndevos-x240.usersys.redhat.com> Message-ID: Hi all, On 10/31/2016 05:20 PM, Ken Dreyer wrote: > full logs: > > https://www.centos.org/minutes/2016/october/centos-devel.2016-10-28-13.05.log.html as I understand log from this meeting Tendrl packages should be in Fedora first and then they can be in CentOS. I've created Jira story https://tendrl.atlassian.net/browse/TEN-131 -- Best Regards, Martin Kudlej. RHSC/USM Senior Quality Assurance Engineer Red Hat Czech s.r.o. Phone: +420 532 294 155 E-mail:mkudlej at redhat.com IRC: mkudlej at #brno, #gluster, #storage-qa, #rhs, #rh-ceph, #usm-meeting @ redhat #tendrl-devel @ freenode From fbalak at redhat.com Tue Nov 1 14:57:41 2016 From: fbalak at redhat.com (Filip Balak) Date: Tue, 1 Nov 2016 10:57:41 -0400 (EDT) Subject: [Tendrl-devel] Configuration files for Tendrl In-Reply-To: <1338748915.5484351.1478011967631.JavaMail.zimbra@redhat.com> Message-ID: <1128115094.5486003.1478012261386.JavaMail.zimbra@redhat.com> Hi Nishanth, Could you please help me with configuration of machines for running tendrl api? I have three machines: Server - there are installed: ETCD, Tendrl/tendrl - IPv4 is server_ip Gluster1 and Gluster2 - there are installed: ETCD, common_bridge, gluster_bridge, node_agent - IPv4 are gluster1_ip and gluster2_ip How should configuration files (/etc/etcd/etcd.conf, /etc/tendrl/tendrl.conf, /config/etcd.yml for Tendrl/tendrl) look like so that I could access API on Server and use Gluster1 and Gluster2 to store Gluster bricks? We have some ansible roles for this on https://github.com/Tendrl/usmqe-setup. As we are asking under the TEN-36 ticket here[1], we need to document the overall tendrl setup. [1] https://tendrl.atlassian.net/browse/TEN-36?focusedCommentId=10949&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-10949 Thank you, Filip Balak From pmcgarry at redhat.com Tue Nov 1 19:43:15 2016 From: pmcgarry at redhat.com (Patrick McGarry) Date: Tue, 1 Nov 2016 15:43:15 -0400 Subject: [Tendrl-devel] Tendrl site In-Reply-To: References: Message-ID: Hey, forgot to mention that I updated the DNS yesterday and tweaked a few things today so it would work a bit better. http://tendrl.org should be up and functional now. Let me know if anything looks amiss. On Thu, Oct 27, 2016 at 11:09 PM, Jeff Applewhite wrote: > Hi All > > We have a basic site up for viewing and feedback. Please take a look at > https://tendrl.github.io/ > > Once we are happy with the final product we can flip the DNS switch and make > tendrl.org live. > > Thanks to Tuomas Kuosmanen for all his work on this! > > -- > Jeff Applewhite > Principal Product Manager > -- Best Regards, Patrick McGarry Director Ceph Community || Red Hat http://ceph.com || http://community.redhat.com @scuttlemonkey || @ceph From japplewh at redhat.com Tue Nov 1 20:07:13 2016 From: japplewh at redhat.com (Jeff Applewhite) Date: Tue, 1 Nov 2016 16:07:13 -0400 Subject: [Tendrl-devel] Tendrl site In-Reply-To: References: Message-ID: Thanks Patrick! On Tue, Nov 1, 2016 at 3:43 PM, Patrick McGarry wrote: > Hey, forgot to mention that I updated the DNS yesterday and tweaked a > few things today so it would work a bit better. > > http://tendrl.org should be up and functional now. Let me know if > anything looks amiss. > > > On Thu, Oct 27, 2016 at 11:09 PM, Jeff Applewhite > wrote: > > Hi All > > > > We have a basic site up for viewing and feedback. Please take a look at > > https://tendrl.github.io/ > > > > Once we are happy with the final product we can flip the DNS switch and > make > > tendrl.org live. > > > > Thanks to Tuomas Kuosmanen for all his work on this! > > > > -- > > Jeff Applewhite > > Principal Product Manager > > > > > > -- > > Best Regards, > > Patrick McGarry > Director Ceph Community || Red Hat > http://ceph.com || http://community.redhat.com > @scuttlemonkey || @ceph > -- Jeff Applewhite Principal Product Manager From nthomas at redhat.com Wed Nov 2 06:29:06 2016 From: nthomas at redhat.com (Nishanth Thomas) Date: Wed, 2 Nov 2016 11:59:06 +0530 Subject: [Tendrl-devel] Provisioning Integration for Ceph and Gluster In-Reply-To: References: Message-ID: On Mon, Oct 31, 2016 at 7:38 PM, Alfredo Deza wrote: > On Wed, Oct 26, 2016 at 11:45 PM, Nishanth Thomas > wrote: > > Hi Alfredo, > > > > Here is the first cut: > > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1319856 - ceph-installer > > executes installation of packages in sequences for different nodes > > https://bugzilla.redhat.com/show_bug.cgi?id=1380628 - Task status is not > > presented in consumable manner and not updated periodically > > https://bugzilla.redhat.com/show_bug.cgi?id=1313935 - properly handle > > ansible-playbook processes that are hung > > https://bugzilla.redhat.com/show_bug.cgi?id=1305269 - > > Shrink a ceph cluster by removing nodes or services using the installer > API > > https://bugzilla.redhat.com/show_bug.cgi?id=1309415 - > > Allow callback URLs on API endpoints > > > > Please have look. We can further discuss on the expectations from our > side > > which will help you to scope out the work. > > What is the timeframe that we are looking at this before code-freeze? > Cluster installation is part of 'Tendrl-2'. So Ideal time to have this available to us is late November. > > > > > Thanks, > > Nishanth > > > > On Wed, Oct 26, 2016 at 9:09 PM, Mrugesh Karnik > > > wrote: > > > >> Hi, > >> > >> We had a discussion with Alfredo Deza (ceph-installer) and Sachidanand > >> (gdeploy) wrt the integration of their respective projects with tendrl's > >> provisioning framework for provisioning ceph and gluster. Here are the > >> takeaways from the discussion. > >> > >> > >> ceph-installer vs ceph-ansible > >> > >> * ceph-installer provides a wrapper over ceph-ansible with normalised > >> output and allows for a more granular execution of the provisioning > flows. > >> * The tendrl team has prior experience with integrating with > ceph-installer > >> in skyrings. > >> * Developing a wrapper over ceph-ansible would take some development > >> effort. However, ceph-installer already addresses some of the problems > that > >> would solved by such a wrapper. > >> > >> We've decided to go ahead with ceph-installer integration because of > these > >> reasons. There are some existing bugs that have been filed against > >> ceph-installer which need to be fixed. > >> > >> > >> tendrl integration > >> > >> * tendrl's provisioning design enables ansible based systems by handling > >> user account creation and ssh access. Both ceph-installer and gdeploy > >> depend upon this functionality. tendrl would ship this functionality as > >> part of it's Native Execution Scenario. > >> * Wrapper scripts of small ansible playbooks would be required to allow > >> tendrl to deploy ceph-installer and gdeploy on it's Provisioning Control > >> Node as part of the Internal Execution Scenario. > >> * The wrapper scripts, along with the system-specific YAML definition > files > >> would enable tendrl to handle provisioning flows. Together these would > >> serve as the Provisioner Interface Modules. > >> > >> > >> Plan of Action: > >> > >> * Nishanth would provide the list of bugs to be fixed in ceph-installer > to > >> Alfredo. > >> * Alfredo would scope out the development effort required to implement > the > >> fixes to the aforementioned bugs. > >> * Mrugesh will work upon the YAML flow definitions, 2nd November > onwards. > >> * Sachidanand, Alfredo and Mrugesh will work together to scope out the > >> exact details of the implementation required to have a working tendrl > >> provisioning integration based on the YAML flows. > >> * Sachidanand and Mrugesh would work upon the integration during the > hack > >> days in Bangalore in the week of 7th November. > >> * Based on the details that would emerge in the week of 7th November, > >> feature requests would be filed against ceph-installer and gdeploy for > >> implementation. > >> > >> -- > >> Mrugesh > >> _______________________________________________ > >> Tendrl-devel mailing list > >> Tendrl-devel at redhat.com > >> https://www.redhat.com/mailman/listinfo/tendrl-devel > >> > > _______________________________________________ > > Tendrl-devel mailing list > > Tendrl-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/tendrl-devel > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From dnarayan at redhat.com Wed Nov 2 07:29:38 2016 From: dnarayan at redhat.com (Darshan Narayana Murthy) Date: Wed, 2 Nov 2016 03:29:38 -0400 (EDT) Subject: [Tendrl-devel] Gluster brick creation via tendrl In-Reply-To: <5bd7de81-7ce1-b7bf-3aed-ba074e5c110d@redhat.com> References: <559141840.4823645.1477662883806.JavaMail.zimbra@redhat.com> <5bd7de81-7ce1-b7bf-3aed-ba074e5c110d@redhat.com> Message-ID: <1206181106.5968305.1478071778432.JavaMail.zimbra@redhat.com> Thank you for your suggestions/pointers, will update this thread after discussion with blivet/Springfield folks. -Darshan ----- Original Message ----- > From: "Ric Wheeler" > To: "Mailing list for the contributors to the Tendrl project" > Sent: Monday, October 31, 2016 6:53:55 PM > Subject: Re: [Tendrl-devel] Gluster brick creation via tendrl > > On 10/31/2016 09:05 AM, Sayan Saha wrote: > > On 10/30/16 11:08 AM, Sankarshan Mukhopadhyay wrote: > >> On Fri, Oct 28, 2016 at 7:24 PM, Darshan Narayana Murthy > >> wrote: > >>> This is regarding the approach to be taken by tendrl for automating > >>> the process of gluster brick creation. I have investigated some of > >>> the tools available like pyudev, storaged, udisks, blivet for this > >>> purpose. > >>> > >>> >From the analysis it seems to me that blivet would be the better > >>> choice for our use case. I have tracked the details of this analysis > >>> in a github issue[1]. > > > > blivet is what we use today with RHGS-C. The RHGS-C team worked with the > > blivet team to make it happen. So we have knowledge about how to leverage > > it > > within the team. > > > > Thanks > > Sayan > > As I mentioned early in the thread, we need to sync up with David who lead > the > blivet team about its future versus the new projects (Springfield) which I > think > is their future direction. > > Ric > > > > >>> > >>> Please, have a look and provide your inputs/comments. > >>> > >>> [1] https://github.com/Tendrl/documentation/issues/49 > >> Thanks for putting together the assessment in an issue. It would be > >> good to reach out to the developers (of blivet) to scope out any known > >> gaps/issues as well as endorsing this choice. We could look to a > >> discussion on the issue itself. > >> > >> Also, you conclude "None of these have capability to detect hardware > >> raid volume related details. I guess good number of customers might be > >> using this. Raid specific details like RAID leval, stripe count, disk > >> count are requirement to provision the bricks considering best > >> practices wrt performance." I believe these are important and evolving > >> data to gather in order to be able to create a profile/shape of the > >> underlying hardware enabling the storage system. > >> > >> Additionally, might be > >> something worth taking a quick look at. > >> > >> > > > > _______________________________________________ > > Tendrl-devel mailing list > > Tendrl-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/tendrl-devel > > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From mbukatov at redhat.com Wed Nov 2 09:35:40 2016 From: mbukatov at redhat.com (Martin Bukatovic) Date: Wed, 2 Nov 2016 10:35:40 +0100 Subject: [Tendrl-devel] SELinux support in Tendrl projects Message-ID: <78cd9161-33af-a48b-99e4-23133d3e376f@redhat.com> Dear all, What is the current plan of SELinux support in Tendrl projects? Based on my previous discussions with Lukas Vrabec from SELinux team in Red Hat, my understanding is that it's best to maintain the SELinux policy directly in the upstream project by upstream developers. Even though we are in beginning of the Tendrl project, I think that it would make sense to draft a plan how we are going to start and develop the SELinux policy for various Tendrl components in upstream directly. We don't need to include the selinux support in the 1st upstream release, but I see a value in starting working on this early. -- Martin Bukatovic USM QE team From deb at redhat.com Wed Nov 2 12:16:52 2016 From: deb at redhat.com (Soumya Deb) Date: Wed, 2 Nov 2016 17:46:52 +0530 Subject: [Tendrl-devel] Make vs Gulp: can we just choose one? Message-ID: Presently (the existing code), we're using both of those for our frontend dev-env. Gulp is only being used in the frontend repo, and make is being used all over. Make is generic and almost "expected" in any build environment, whereas Gulp has a NodeJS dependency. On the other hand, Gulp is a bit sweeter and verbose (syntactically) than make for frontend project (has it's own flavor of DSL). Now, I personally don't see the point of having/needing to use both, but selecting only either has the risk of pissing off some people regardless. Can we make some solid argument to move ahead with one, instead of both? FWIW, I find no reason that we can't just go with make only. So far, I haven't found it's incapable of doing anything that gulp is able to for our purpose (but I might be wrong). It also ensures less dependency overheads (and is presumably faster). From mbukatov at redhat.com Wed Nov 2 14:53:58 2016 From: mbukatov at redhat.com (Martin Bukatovic) Date: Wed, 2 Nov 2016 15:53:58 +0100 Subject: [Tendrl-devel] Make vs Gulp: can we just choose one? In-Reply-To: References: Message-ID: <32b1574e-4ef8-608c-18b2-eb5866d6fcee@redhat.com> On 11/02/2016 01:16 PM, Soumya Deb wrote: > Presently (the existing code), we're using both of those for our frontend > dev-env. > > Gulp is only being used in the frontend repo, and make is being used all > over. Make is generic and almost "expected" in any build environment, > whereas Gulp has a NodeJS dependency. On the other hand, Gulp is a bit > sweeter and verbose (syntactically) than make for frontend project (has > it's own flavor of DSL). > > Now, I personally don't see the point of having/needing to use both, but > selecting only either has the risk of pissing off some people regardless. > > Can we make some solid argument to move ahead with one, instead of both? > > FWIW, I find no reason that we can't just go with make only. So far, I > haven't found it's incapable of doing anything that gulp is able to for our > purpose (but I might be wrong). It also ensures less dependency overheads > (and is presumably faster). Just to make your suggestion clear: I assume you are not proposing other non javascript components to use gulp, which would be hard to justify. I rather read it that you are not satisfied with the fact that in the frontend project, which is written in javascript, both make and gulp tools are used. Do I understand it right? -- Martin Bukatovic USM QE team From kdreyer at redhat.com Wed Nov 2 16:08:04 2016 From: kdreyer at redhat.com (Ken Dreyer) Date: Wed, 2 Nov 2016 10:08:04 -0600 Subject: [Tendrl-devel] CentOS Storage SIG meeting on Friday 28-Oct-2016 at 13:00 UTC In-Reply-To: References: <20161026185118.GD2936@ndevos-x240.usersys.redhat.com> <20161027165114.GG2936@ndevos-x240.usersys.redhat.com> Message-ID: On Tue, Nov 1, 2016 at 3:24 AM, Martin Kudlej wrote: > Hi all, > > On 10/31/2016 05:20 PM, Ken Dreyer wrote: >> >> full logs: >> >> >> https://www.centos.org/minutes/2016/october/centos-devel.2016-10-28-13.05.log.html > > > as I understand log from this meeting Tendrl packages should be in Fedora > first and then they can be in CentOS. > I've created Jira story https://tendrl.atlassian.net/browse/TEN-131 > Getting the packages into Fedora will take some time. To keep things moving, I think we should still host all the upstream builds "somewhere" in the meantime. For the ceph-installer stack, I've put the el7 builds into https://copr.fedorainfracloud.org/coprs/ktdreyer/ceph-installer/ We could use a separate Copr for Tendrl, and add all developers' FAS usernames to have permissions to add builds to it. Nishanth, want to create that? You can do it at https://copr.fedorainfracloud.org/coprs/ - Ken From kdreyer at redhat.com Wed Nov 2 16:14:01 2016 From: kdreyer at redhat.com (Ken Dreyer) Date: Wed, 2 Nov 2016 10:14:01 -0600 Subject: [Tendrl-devel] what are the Ubuntu agent packages? Message-ID: >From what I understand, Tendrl has some "agent" packages that must run on each Ceph node? What are the Tendrl agent packages that we will need to create for Ubuntu Ceph clusters? - Ken From kanagaraj.ktr at gmail.com Wed Nov 2 17:29:00 2016 From: kanagaraj.ktr at gmail.com (Kanagaraj M) Date: Wed, 2 Nov 2016 22:59:00 +0530 Subject: [Tendrl-devel] Make vs Gulp: can we just choose one? In-Reply-To: <32b1574e-4ef8-608c-18b2-eb5866d6fcee@redhat.com> References: <32b1574e-4ef8-608c-18b2-eb5866d6fcee@redhat.com> Message-ID: Though both make and gulp are build systems, they serve different purpose. Since the project already has a dependency on node ecosystem, so gulp relying on node is not something we need to worry about. In the existing code, gulp helps in 1).compiling TS to JS 2) create sourcemaps 3) Combine JS files 4) Minify them 5) Copy to relevant path 6) Inject the script tags in html, and lot more. The above can be achieved using Makefile, but i don't think its going to be as straight forward as using gulp(or any nodejs based build system). IMO, using Makfile for all kind of build tasks are not the way to go. Instead you can move from gulp+browserify to webpack and npm scripts if you think gulp is verbose and complex. Then use the Makefile for packaging and release. Thanks, Kanagaraj On Wed, Nov 2, 2016 at 8:23 PM, Martin Bukatovic wrote: > On 11/02/2016 01:16 PM, Soumya Deb wrote: > > Presently (the existing code), we're using both of those for our frontend > > dev-env. > > > > Gulp is only being used in the frontend repo, and make is being used all > > over. Make is generic and almost "expected" in any build environment, > > whereas Gulp has a NodeJS dependency. On the other hand, Gulp is a bit > > sweeter and verbose (syntactically) than make for frontend project (has > > it's own flavor of DSL). > > > > Now, I personally don't see the point of having/needing to use both, but > > selecting only either has the risk of pissing off some people regardless. > > > > Can we make some solid argument to move ahead with one, instead of both? > > > > FWIW, I find no reason that we can't just go with make only. So far, I > > haven't found it's incapable of doing anything that gulp is able to for > our > > purpose (but I might be wrong). It also ensures less dependency overheads > > (and is presumably faster). > > Just to make your suggestion clear: I assume you are not proposing other > non javascript components to use gulp, which would be hard to justify. > I rather read it that you are not satisfied with the fact that in the > frontend project, which is written in javascript, both make and gulp > tools are used. Do I understand it right? > > -- > Martin Bukatovic > USM QE team > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From julim at redhat.com Wed Nov 2 18:27:20 2016 From: julim at redhat.com (Ju Lim) Date: Wed, 2 Nov 2016 14:27:20 -0400 Subject: [Tendrl-devel] Tendrl Sprint 3 Demo Details (Recording & Notes) Message-ID: Team: The demo recording from the Sprint 3 demo is now available along with the agenda and notes (feedback and Q&A). They can be found at Sprint 003 Demo Agenda & Notes . If folks have any comments and/or feedback on the demo(s), please reply all to this email thread. As always, all Sprint goals, demo agenda, demo recording, and notes may be found at the Tendrl Sprints Landing Page , which is an index of all sprints to-date. Thank you, Ju From kanagaraj.ktr at gmail.com Thu Nov 3 03:55:21 2016 From: kanagaraj.ktr at gmail.com (Kanagaraj M) Date: Thu, 3 Nov 2016 09:25:21 +0530 Subject: [Tendrl-devel] Tendrl Sprint 3 Demo Details (Recording & Notes) In-Reply-To: References: Message-ID: The demo looks nice. I have few questions regarding the frontend part. 1. Does this repo https://github.com/Tendrl/tendrl_frontend has the code(or PRs) for whatever demoed on frontend? 2. Dynamic generation of UI is good. But will this work for complex workflow? From the ux design of volume/fileshare creation https://redhat.invisionapp.com/share/Q78YMAVDJ#/screens , it is lot more than a single form. This is just an example, there might be more screens like this. Karnan has already raised this question https://www.redhat.com/archives/tendrl-devel/2016-October/msg00119.html Thanks, Kanagaraj On Wed, Nov 2, 2016 at 11:57 PM, Ju Lim wrote: > Team: > > The demo recording from the Sprint 3 demo is now available along with the > agenda and notes (feedback and Q&A). They can be found at Sprint 003 Demo > Agenda & Notes > . > > If folks have any comments and/or feedback on the demo(s), please reply all > to this email thread. > > As always, all Sprint goals, demo agenda, demo recording, and notes may be > found at the Tendrl Sprints Landing Page > , which is an index > of all sprints to-date. > > Thank you, > Ju > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From shtripat at redhat.com Thu Nov 3 12:03:05 2016 From: shtripat at redhat.com (Shubhendu Tripathi) Date: Thu, 3 Nov 2016 17:33:05 +0530 Subject: [Tendrl-devel] PRs for review Message-ID: <2a7dc921-9f79-804e-cd2a-414d95ba84e1@redhat.com> Hi Team, Below are the PRs which need reviews (these are only my PRs ;). Please spend sometime for this. 1. https://github.com/Tendrl/gluster_bridge/pull/30 2. https://github.com/Tendrl/gluster_bridge/pull/31 3. https://github.com/Tendrl/gluster_bridge/pull/32 4. https://github.com/Tendrl/gluster_bridge/pull/39 5. https://github.com/Tendrl/ceph_bridge/pull/16 Thanks and Regards, Shubhendu From sankarshan.mukhopadhyay at gmail.com Thu Nov 3 15:24:22 2016 From: sankarshan.mukhopadhyay at gmail.com (Sankarshan Mukhopadhyay) Date: Thu, 3 Nov 2016 20:54:22 +0530 Subject: [Tendrl-devel] Gluster brick creation via tendrl In-Reply-To: <5bd7de81-7ce1-b7bf-3aed-ba074e5c110d@redhat.com> References: <559141840.4823645.1477662883806.JavaMail.zimbra@redhat.com> <5bd7de81-7ce1-b7bf-3aed-ba074e5c110d@redhat.com> Message-ID: [top-posting to maintain a bit of context and I've copied David in for additional comments] A while ago a few members from the Tendrl development team met with David to work through the aspect of choice of blivet to undertake the task Darshan has assessed it for. During this David suggested that a combination of libstoragemgmt and blivet is a good stack to consider for short and medium term. Over the period of the coming year, there are a couple of features landing up in storaged and blivet which would make this a more suitable choice to turn towards. libstoragemgmt has Python bindings and thus language specific issues are not immediately obvious. There's certain set of proof-of-concept implementation around libstoragemgmt which can be put together to interact with the Tendrl node agents. We'll continue that conversation with David and Gris. The hardware detection, information etc layer is seeing a convergence of tooling and this is something which has immense utility for Tendrl to consume. /s On Mon, Oct 31, 2016 at 6:53 PM, Ric Wheeler wrote: > On 10/31/2016 09:05 AM, Sayan Saha wrote: >> >> On 10/30/16 11:08 AM, Sankarshan Mukhopadhyay wrote: >>> >>> On Fri, Oct 28, 2016 at 7:24 PM, Darshan Narayana Murthy >>> wrote: >>>> >>>> This is regarding the approach to be taken by tendrl for automating >>>> the process of gluster brick creation. I have investigated some of >>>> the tools available like pyudev, storaged, udisks, blivet for this >>>> purpose. >>>> >>>> >From the analysis it seems to me that blivet would be the better >>>> choice for our use case. I have tracked the details of this analysis >>>> in a github issue[1]. >> >> >> blivet is what we use today with RHGS-C. The RHGS-C team worked with the >> blivet team to make it happen. So we have knowledge about how to leverage it >> within the team. >> >> Thanks >> Sayan > > > As I mentioned early in the thread, we need to sync up with David who lead > the blivet team about its future versus the new projects (Springfield) which > I think is their future direction. > > Ric > > >> >>>> >>>> Please, have a look and provide your inputs/comments. >>>> >>>> [1] https://github.com/Tendrl/documentation/issues/49 >>> >>> Thanks for putting together the assessment in an issue. It would be >>> good to reach out to the developers (of blivet) to scope out any known >>> gaps/issues as well as endorsing this choice. We could look to a >>> discussion on the issue itself. >>> >>> Also, you conclude "None of these have capability to detect hardware >>> raid volume related details. I guess good number of customers might be >>> using this. Raid specific details like RAID leval, stripe count, disk >>> count are requirement to provision the bricks considering best >>> practices wrt performance." I believe these are important and evolving >>> data to gather in order to be able to create a profile/shape of the >>> underlying hardware enabling the storage system. >>> >>> Additionally, might be >>> something worth taking a quick look at. -- sankarshan mukhopadhyay From nthomas at redhat.com Fri Nov 4 06:39:14 2016 From: nthomas at redhat.com (Nishanth Thomas) Date: Fri, 4 Nov 2016 12:09:14 +0530 Subject: [Tendrl-devel] CentOS Storage SIG meeting on Friday 28-Oct-2016 at 13:00 UTC In-Reply-To: References: <20161026185118.GD2936@ndevos-x240.usersys.redhat.com> <20161027165114.GG2936@ndevos-x240.usersys.redhat.com> Message-ID: On Wed, Nov 2, 2016 at 9:38 PM, Ken Dreyer wrote: > On Tue, Nov 1, 2016 at 3:24 AM, Martin Kudlej wrote: > > Hi all, > > > > On 10/31/2016 05:20 PM, Ken Dreyer wrote: > >> > >> full logs: > >> > >> > >> https://www.centos.org/minutes/2016/october/centos- > devel.2016-10-28-13.05.log.html > > > > > > as I understand log from this meeting Tendrl packages should be in Fedora > > first and then they can be in CentOS. > > I've created Jira story https://tendrl.atlassian.net/browse/TEN-131 > > > > Getting the packages into Fedora will take some time. > > To keep things moving, I think we should still host all the upstream > builds "somewhere" in the meantime. > > For the ceph-installer stack, I've put the el7 builds into > https://copr.fedorainfracloud.org/coprs/ktdreyer/ceph-installer/ > > We could use a separate Copr for Tendrl, and add all developers' FAS > usernames to have permissions to add builds to it. Nishanth, want to > create that? You can do it at https://copr.fedorainfracloud.org/coprs/ Thanks Ken. Its a good option until the fedora packages are ready. I will take care of this. > > > - Ken > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From nthomas at redhat.com Fri Nov 4 07:12:02 2016 From: nthomas at redhat.com (Nishanth Thomas) Date: Fri, 4 Nov 2016 12:42:02 +0530 Subject: [Tendrl-devel] Gluster brick creation via tendrl In-Reply-To: References: <559141840.4823645.1477662883806.JavaMail.zimbra@redhat.com> <5bd7de81-7ce1-b7bf-3aed-ba074e5c110d@redhat.com> Message-ID: Is there a recording available ? On Thu, Nov 3, 2016 at 8:54 PM, Sankarshan Mukhopadhyay < sankarshan.mukhopadhyay at gmail.com> wrote: > [top-posting to maintain a bit of context and I've copied David in for > additional comments] > > A while ago a few members from the Tendrl development team met with > David to work through the aspect of choice of blivet to undertake the > task Darshan has assessed it for. During this David suggested that a > combination of libstoragemgmt and blivet is a good stack to consider > for short and medium term. Over the period of the coming year, there > are a couple of features landing up in storaged and blivet which would > make this a more suitable choice to turn towards. > > libstoragemgmt has Python bindings and thus language specific issues > are not immediately obvious. There's certain set of proof-of-concept > implementation around libstoragemgmt which can be put together to > interact with the Tendrl node agents. We'll continue that conversation > with David and Gris. > > The hardware detection, information etc layer is seeing a convergence > of tooling and this is something which has immense utility for Tendrl > to consume. > > /s > > On Mon, Oct 31, 2016 at 6:53 PM, Ric Wheeler wrote: > > On 10/31/2016 09:05 AM, Sayan Saha wrote: > >> > >> On 10/30/16 11:08 AM, Sankarshan Mukhopadhyay wrote: > >>> > >>> On Fri, Oct 28, 2016 at 7:24 PM, Darshan Narayana Murthy > >>> wrote: > >>>> > >>>> This is regarding the approach to be taken by tendrl for automating > >>>> the process of gluster brick creation. I have investigated some of > >>>> the tools available like pyudev, storaged, udisks, blivet for this > >>>> purpose. > >>>> > >>>> >From the analysis it seems to me that blivet would be the better > >>>> choice for our use case. I have tracked the details of this analysis > >>>> in a github issue[1]. > >> > >> > >> blivet is what we use today with RHGS-C. The RHGS-C team worked with the > >> blivet team to make it happen. So we have knowledge about how to > leverage it > >> within the team. > >> > >> Thanks > >> Sayan > > > > > > As I mentioned early in the thread, we need to sync up with David who > lead > > the blivet team about its future versus the new projects (Springfield) > which > > I think is their future direction. > > > > Ric > > > > > >> > >>>> > >>>> Please, have a look and provide your inputs/comments. > >>>> > >>>> [1] https://github.com/Tendrl/documentation/issues/49 > >>> > >>> Thanks for putting together the assessment in an issue. It would be > >>> good to reach out to the developers (of blivet) to scope out any known > >>> gaps/issues as well as endorsing this choice. We could look to a > >>> discussion on the issue itself. > >>> > >>> Also, you conclude "None of these have capability to detect hardware > >>> raid volume related details. I guess good number of customers might be > >>> using this. Raid specific details like RAID leval, stripe count, disk > >>> count are requirement to provision the bricks considering best > >>> practices wrt performance." I believe these are important and evolving > >>> data to gather in order to be able to create a profile/shape of the > >>> underlying hardware enabling the storage system. > >>> > >>> Additionally, might be > >>> something worth taking a quick look at. > > > > > -- > sankarshan mukhopadhyay > > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From shtripat at redhat.com Fri Nov 4 07:57:36 2016 From: shtripat at redhat.com (Shubhendu Tripathi) Date: Fri, 4 Nov 2016 13:27:36 +0530 Subject: [Tendrl-devel] Gluster brick creation via tendrl In-Reply-To: References: <559141840.4823645.1477662883806.JavaMail.zimbra@redhat.com> <5bd7de81-7ce1-b7bf-3aed-ba074e5c110d@redhat.com> Message-ID: <11cdcfae-f1cf-1b18-08aa-4542c845447f@redhat.com> Jeff shared already and its at https://bluejeans.com/s/kMuhs/ Regards, Shubhendu On 11/04/2016 12:42 PM, Nishanth Thomas wrote: > Is there a recording available ? > > On Thu, Nov 3, 2016 at 8:54 PM, Sankarshan Mukhopadhyay < > sankarshan.mukhopadhyay at gmail.com> wrote: > >> [top-posting to maintain a bit of context and I've copied David in for >> additional comments] >> >> A while ago a few members from the Tendrl development team met with >> David to work through the aspect of choice of blivet to undertake the >> task Darshan has assessed it for. During this David suggested that a >> combination of libstoragemgmt and blivet is a good stack to consider >> for short and medium term. Over the period of the coming year, there >> are a couple of features landing up in storaged and blivet which would >> make this a more suitable choice to turn towards. >> >> libstoragemgmt has Python bindings and thus language specific issues >> are not immediately obvious. There's certain set of proof-of-concept >> implementation around libstoragemgmt which can be put together to >> interact with the Tendrl node agents. We'll continue that conversation >> with David and Gris. >> >> The hardware detection, information etc layer is seeing a convergence >> of tooling and this is something which has immense utility for Tendrl >> to consume. >> >> /s >> >> On Mon, Oct 31, 2016 at 6:53 PM, Ric Wheeler wrote: >>> On 10/31/2016 09:05 AM, Sayan Saha wrote: >>>> On 10/30/16 11:08 AM, Sankarshan Mukhopadhyay wrote: >>>>> On Fri, Oct 28, 2016 at 7:24 PM, Darshan Narayana Murthy >>>>> wrote: >>>>>> This is regarding the approach to be taken by tendrl for automating >>>>>> the process of gluster brick creation. I have investigated some of >>>>>> the tools available like pyudev, storaged, udisks, blivet for this >>>>>> purpose. >>>>>> >>>>>> >From the analysis it seems to me that blivet would be the better >>>>>> choice for our use case. I have tracked the details of this analysis >>>>>> in a github issue[1]. >>>> >>>> blivet is what we use today with RHGS-C. The RHGS-C team worked with the >>>> blivet team to make it happen. So we have knowledge about how to >> leverage it >>>> within the team. >>>> >>>> Thanks >>>> Sayan >>> >>> As I mentioned early in the thread, we need to sync up with David who >> lead >>> the blivet team about its future versus the new projects (Springfield) >> which >>> I think is their future direction. >>> >>> Ric >>> >>> >>>>>> Please, have a look and provide your inputs/comments. >>>>>> >>>>>> [1] https://github.com/Tendrl/documentation/issues/49 >>>>> Thanks for putting together the assessment in an issue. It would be >>>>> good to reach out to the developers (of blivet) to scope out any known >>>>> gaps/issues as well as endorsing this choice. We could look to a >>>>> discussion on the issue itself. >>>>> >>>>> Also, you conclude "None of these have capability to detect hardware >>>>> raid volume related details. I guess good number of customers might be >>>>> using this. Raid specific details like RAID leval, stripe count, disk >>>>> count are requirement to provision the bricks considering best >>>>> practices wrt performance." I believe these are important and evolving >>>>> data to gather in order to be able to create a profile/shape of the >>>>> underlying hardware enabling the storage system. >>>>> >>>>> Additionally, might be >>>>> something worth taking a quick look at. >> >> >> >> -- >> sankarshan mukhopadhyay >> >> >> _______________________________________________ >> Tendrl-devel mailing list >> Tendrl-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/tendrl-devel >> > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From shtripat at redhat.com Fri Nov 4 08:19:34 2016 From: shtripat at redhat.com (Shubhendu Tripathi) Date: Fri, 4 Nov 2016 13:49:34 +0530 Subject: [Tendrl-devel] Need clarification on task https://tendrl.atlassian.net/browse/TEN-135. Please respond Message-ID: <920f6474-fd3d-a478-3046-e98daa2460b6@redhat.com> From mrugesh at brainfunked.org Fri Nov 4 09:05:43 2016 From: mrugesh at brainfunked.org (Mrugesh Karnik) Date: Fri, 4 Nov 2016 14:35:43 +0530 Subject: [Tendrl-devel] Need clarification on task https://tendrl.atlassian.net/browse/TEN-135. Please respond In-Reply-To: <920f6474-fd3d-a478-3046-e98daa2460b6@redhat.com> References: <920f6474-fd3d-a478-3046-e98daa2460b6@redhat.com> Message-ID: The pre and post conditions are to be used for verification. The pre condition is to ensure that the object being modified as part of an action hasn't changed state since the action was originally invoked from the API. The post condition is to check that the object state is what's expected as the result of the action being carried out. For example, on an expand operation that asks for +5GB on an object 10GB in size, the pre condition would be to ensure that the object's size is still 10GB before carrying the action out. Post condition would be to check that the size after the expansion is the expected value, in this case 15GB. -- Mrugesh From mrugesh at brainfunked.org Fri Nov 4 09:46:19 2016 From: mrugesh at brainfunked.org (Mrugesh Karnik) Date: Fri, 4 Nov 2016 15:16:19 +0530 Subject: [Tendrl-devel] Tendrl Sprint 3 Demo Details (Recording & Notes) In-Reply-To: References: Message-ID: On 3 November 2016 at 09:25, Kanagaraj M wrote: > 2. Dynamic generation of UI is good. But will this work for complex > workflow? From the ux design of volume/fileshare creation > https://redhat.invisionapp.com/share/Q78YMAVDJ#/screens , it is lot more > than a single form. This is just an example, there might be more screens > like this. Karnan has already raised this question > https://www.redhat.com/archives/tendrl-devel/2016-October/msg00119.html I don't think that it'll be possible to make the UI completely dynamic. There are actions that could be generated dynamically, based on the API. These could be templatised. The workflows you're talking about require a certain level of hard-coding and also support for multi-step operations from the API itself. The workflows could then be defined in the API, but that still leaves some level of hard-coding to be done in the UI. This hard-coding would need to address UI-specific text, layout etc. which cannot be addressed by the API. The way I'm looking at this is that UI, like our backend eventually would need to serve as a framework. The main UI codebase itself could be completely generic. We could then add plugins specific to the storage systems which would allow a certain level of hard-coding to tie the API responses to specific layouts to be rendered. This isn't a very straightforward coding exercise though. So the approach, I think, that'll be the most beneficial is to approach the specific UI related work based on the features enabled via the API. It's OK to not be generic in the beginning. However, as we go along, it should be possible to start abstracting out the UI code to be split into the generic framework and hard-coded plugins. -- Mrugesh From deb at redhat.com Fri Nov 4 10:12:49 2016 From: deb at redhat.com (Soumya Deb) Date: Fri, 4 Nov 2016 15:42:49 +0530 Subject: [Tendrl-devel] Make vs Gulp: can we just choose one? In-Reply-To: <32b1574e-4ef8-608c-18b2-eb5866d6fcee@redhat.com> References: <32b1574e-4ef8-608c-18b2-eb5866d6fcee@redhat.com> Message-ID: On 2 November 2016 at 20:23, Martin Bukatovic wrote: > On 11/02/2016 01:16 PM, Soumya Deb wrote: > > Presently (the existing code), we're using both of those for our frontend > > dev-env. > > > > Gulp is only being used in the frontend repo, and make is being used all > > over. Make is generic and almost "expected" in any build environment, > > whereas Gulp has a NodeJS dependency. On the other hand, Gulp is a bit > > sweeter and verbose (syntactically) than make for frontend project (has > > it's own flavor of DSL). > > > > Now, I personally don't see the point of having/needing to use both, but > > selecting only either has the risk of pissing off some people regardless. > > > > Can we make some solid argument to move ahead with one, instead of both? > > > > FWIW, I find no reason that we can't just go with make only. So far, I > > haven't found it's incapable of doing anything that gulp is able to for > our > > purpose (but I might be wrong). It also ensures less dependency overheads > > (and is presumably faster). > > Just to make your suggestion clear: I assume you are not proposing other > non javascript components to use gulp, which would be hard to justify. > Make, in turn, is using some gulp tasks (which can be done by make itself). Don't know what to think of it, or to justify. > I rather read it that you are not satisfied with the fact that in the > frontend project, which is written in javascript, both make and gulp > tools are used. Do I understand it right? > About 80% agreed. This being a frontend project has little to do with task runner. My trouble is to create and manage and rig two different sets of task runners, which can (and better be) done by one. From shtripat at redhat.com Fri Nov 4 10:46:40 2016 From: shtripat at redhat.com (Shubhendu Tripathi) Date: Fri, 4 Nov 2016 16:16:40 +0530 Subject: [Tendrl-devel] Need clarification on task https://tendrl.atlassian.net/browse/TEN-135. Please respond In-Reply-To: References: <920f6474-fd3d-a478-3046-e98daa2460b6@redhat.com> Message-ID: <66b071f7-96aa-e085-c4aa-d3194ff419dc@redhat.com> Mrugesh, I think I pretty well understand the use of pre and post runs defined for atoms and flows. But, my question is more specific to which component does this validation whether APP or the SDS bridge. Also, I want a confirmation if we need to pass these pre and post run details as part of job (while flow execution), if the second option above is to be opted for. Regards, Shubhendu On 11/04/2016 02:35 PM, Mrugesh Karnik wrote: > The pre and post conditions are to be used for verification. > > The pre condition is to ensure that the object being modified as part of an > action hasn't changed state since the action was originally invoked from > the API. > > The post condition is to check that the object state is what's expected as > the result of the action being carried out. > > For example, on an expand operation that asks for +5GB on an object 10GB in > size, the pre condition would be to ensure that the object's size is still > 10GB before carrying the action out. Post condition would be to check that > the size after the expansion is the expected value, in this case 15GB. > > -- > Mrugesh > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From mkudlej at redhat.com Fri Nov 4 10:50:44 2016 From: mkudlej at redhat.com (Martin Kudlej) Date: Fri, 4 Nov 2016 11:50:44 +0100 Subject: [Tendrl-devel] Jenkins jobs and Ansible playbooks for CentOS CI Message-ID: Hi all, I've prepared Ansible playbooks(thanks Martin B for help) for getting and returning nodes from CentOS CI machine pool. https://github.com/mkudlej/usmqe-centos-ci/tree/master/ansible Also there are jobs for JJB for CentOS CI https://github.com/mkudlej/usmqe-centos-ci/tree/master/jobs If you would like to add Jenkins jobs for building packages, please add them into this repository and use JJB for managing such jobs. Also there is view in Jenkins for Tendrl https://ci.centos.org/view/tendrl/ There are jobs only for getting and returning machines from/to pool. Jobs for testing will be stored in different repository because we can reuse them also in other CIs(for example I expect that Ken can use jobs for testing in Ceph CI). Unfortunately there is no space now in homes in CentOS CI machine, so I cannot test those jobs. Feel free to contact me in case of any questions. -- Best Regards, Martin Kudlej. RHSC/USM Senior Quality Assurance Engineer Red Hat Czech s.r.o. Phone: +420 532 294 155 E-mail:mkudlej at redhat.com IRC: mkudlej at #brno, #gluster, #storage-qa, #rhs, #rh-ceph, #usm-meeting @ redhat #tendrl-devel @ freenode From deb at redhat.com Fri Nov 4 12:37:10 2016 From: deb at redhat.com (Soumya Deb) Date: Fri, 4 Nov 2016 18:07:10 +0530 Subject: [Tendrl-devel] Make vs Gulp: can we just choose one? In-Reply-To: References: <32b1574e-4ef8-608c-18b2-eb5866d6fcee@redhat.com> Message-ID: On 2 November 2016 at 22:59, Kanagaraj M wrote: > Though both make and gulp are build systems, they serve different purpose. > Please do elaborate. I personally don't see them thrive to achieve different goals albeit using different mechanism. > Since the project already has a dependency on node ecosystem, so gulp > relying on node is not something we need to worry about. > "already has node dependency" and gulp is about 75% reason behind that. If we can remove gulp, we are pretty close to not having Node.js deps. I am trying to understand which (if at all) sacrifices are needed for that. > In the existing code, gulp helps in 1).compiling TS to JS 2) create > sourcemaps 3) Combine JS files 4) Minify them 5) Copy to relevant path 6) > Inject the script tags in html, and lot more. > Make can do as well (in fact, I'm trying to find if make can't do something"). > The above can be achieved using Makefile, but i don't think its going to be > as straight forward as using gulp(or any nodejs based build system). > Example, gulp: =========== gulp.task('sass', function () { return gulp.src(path.styles) .pipe($.sourcemaps.init()) .pipe(sass()) .pipe($.concat('tendrl.css')) .pipe($.sourcemaps.write('.')) .pipe(gulp.dest(path.dist + '/css')); Same thing, make: ============= sass: ${SASS} cat $^ | sass --sourcemap compressed > ${DIST}/css/tendrl.css To me, second one is much more straight forward and more versatile. It's leaner and faster as well, as it's executing the commands directly instead of being wrapped within node-modules that does the proxying. > IMO, using Makfile for all kind of build tasks are not the way to go. > Instead you can move from gulp+browserify to webpack and npm scripts if you > think gulp is verbose and complex. Then use the Makefile for packaging and > release. Gulp isn't complex, and switching between gulp/grunt/npm-script is the complete anti-theory of my proposal, unless we're using only one solution. And make is years proven and well supported tool to achieve the same thing - do we need these alternatives? In simpler words, what do we miss out if we move to `make` only? From adeza at redhat.com Fri Nov 4 20:21:13 2016 From: adeza at redhat.com (Alfredo Deza) Date: Fri, 4 Nov 2016 16:21:13 -0400 Subject: [Tendrl-devel] Provisioning Integration for Ceph and Gluster In-Reply-To: References: Message-ID: On Wed, Nov 2, 2016 at 2:29 AM, Nishanth Thomas wrote: > On Mon, Oct 31, 2016 at 7:38 PM, Alfredo Deza wrote: > >> On Wed, Oct 26, 2016 at 11:45 PM, Nishanth Thomas >> wrote: >> > Hi Alfredo, >> > >> > Here is the first cut: >> > We estimate that delivering all of the BZs isn't going to be possible by the end of November. Description on effort per BZ inline. >> > These ones would require significant changes in the installer and in the queue system that the installer supports. It wouldn't be possible to complete these in time >> > https://bugzilla.redhat.com/show_bug.cgi?id=1319856 - ceph-installer >> > executes installation of packages in sequences for different nodes >> > https://bugzilla.redhat.com/show_bug.cgi?id=1380628 - Task status is not >> > presented in consumable manner and not updated periodically This one is tricky to do correctly. We could allow for a timeout value coming from the client. Ceph has numerous spots where it will hang indefinitely regardless of the invocating tool. >> > https://bugzilla.redhat.com/show_bug.cgi?id=1313935 - properly handle >> > ansible-playbook processes that are hung There is already support for this one upstream, but little to no testing. We don't think we can get this in time by EOM. >> > https://bugzilla.redhat.com/show_bug.cgi?id=1305269 - >> > Shrink a ceph cluster by removing nodes or services using the installer >> API This is the one BZ we can actually get in. All API endpoints would allow a new key/value format to define where/what callback to be defined. >> > https://bugzilla.redhat.com/show_bug.cgi?id=1309415 - >> > Allow callback URLs on API endpoints >> > >> > Please have look. We can further discuss on the expectations from our >> side >> > which will help you to scope out the work. Let us know if it is feasible to get things working on your end with just the callback system delivered so that we can start planing for that effort. >> >> What is the timeframe that we are looking at this before code-freeze? >> > > Cluster installation is part of 'Tendrl-2'. So Ideal time to have this > available to us is late November. > > >> >> > >> > Thanks, >> > Nishanth >> > >> > On Wed, Oct 26, 2016 at 9:09 PM, Mrugesh Karnik > > >> > wrote: >> > >> >> Hi, >> >> >> >> We had a discussion with Alfredo Deza (ceph-installer) and Sachidanand >> >> (gdeploy) wrt the integration of their respective projects with tendrl's >> >> provisioning framework for provisioning ceph and gluster. Here are the >> >> takeaways from the discussion. >> >> >> >> >> >> ceph-installer vs ceph-ansible >> >> >> >> * ceph-installer provides a wrapper over ceph-ansible with normalised >> >> output and allows for a more granular execution of the provisioning >> flows. >> >> * The tendrl team has prior experience with integrating with >> ceph-installer >> >> in skyrings. >> >> * Developing a wrapper over ceph-ansible would take some development >> >> effort. However, ceph-installer already addresses some of the problems >> that >> >> would solved by such a wrapper. >> >> >> >> We've decided to go ahead with ceph-installer integration because of >> these >> >> reasons. There are some existing bugs that have been filed against >> >> ceph-installer which need to be fixed. >> >> >> >> >> >> tendrl integration >> >> >> >> * tendrl's provisioning design enables ansible based systems by handling >> >> user account creation and ssh access. Both ceph-installer and gdeploy >> >> depend upon this functionality. tendrl would ship this functionality as >> >> part of it's Native Execution Scenario. >> >> * Wrapper scripts of small ansible playbooks would be required to allow >> >> tendrl to deploy ceph-installer and gdeploy on it's Provisioning Control >> >> Node as part of the Internal Execution Scenario. >> >> * The wrapper scripts, along with the system-specific YAML definition >> files >> >> would enable tendrl to handle provisioning flows. Together these would >> >> serve as the Provisioner Interface Modules. >> >> >> >> >> >> Plan of Action: >> >> >> >> * Nishanth would provide the list of bugs to be fixed in ceph-installer >> to >> >> Alfredo. >> >> * Alfredo would scope out the development effort required to implement >> the >> >> fixes to the aforementioned bugs. >> >> * Mrugesh will work upon the YAML flow definitions, 2nd November >> onwards. >> >> * Sachidanand, Alfredo and Mrugesh will work together to scope out the >> >> exact details of the implementation required to have a working tendrl >> >> provisioning integration based on the YAML flows. >> >> * Sachidanand and Mrugesh would work upon the integration during the >> hack >> >> days in Bangalore in the week of 7th November. >> >> * Based on the details that would emerge in the week of 7th November, >> >> feature requests would be filed against ceph-installer and gdeploy for >> >> implementation. >> >> >> >> -- >> >> Mrugesh >> >> _______________________________________________ >> >> Tendrl-devel mailing list >> >> Tendrl-devel at redhat.com >> >> https://www.redhat.com/mailman/listinfo/tendrl-devel >> >> >> > _______________________________________________ >> > Tendrl-devel mailing list >> > Tendrl-devel at redhat.com >> > https://www.redhat.com/mailman/listinfo/tendrl-devel >> >> _______________________________________________ >> Tendrl-devel mailing list >> Tendrl-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/tendrl-devel >> > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From kanagaraj.ktr at gmail.com Mon Nov 7 09:57:10 2016 From: kanagaraj.ktr at gmail.com (Kanagaraj M) Date: Mon, 7 Nov 2016 15:27:10 +0530 Subject: [Tendrl-devel] Make vs Gulp: can we just choose one? In-Reply-To: References: <32b1574e-4ef8-608c-18b2-eb5866d6fcee@redhat.com> Message-ID: On Fri, Nov 4, 2016 at 6:07 PM, Soumya Deb wrote: > On 2 November 2016 at 22:59, Kanagaraj M wrote: > > > Though both make and gulp are build systems, they serve different > purpose. > > > > Please do elaborate. I personally don't see them thrive to achieve > different goals > albeit using different mechanism. > Gulp is designed to more suitable for a frontend project where the source would go through multiple transformations to reach the dist. > > > > > Since the project already has a dependency on node ecosystem, so gulp > > relying on node is not something we need to worry about. > > > > "already has node dependency" and gulp is about 75% reason behind that. > If we can remove gulp, we are pretty close to not having Node.js deps. > I am trying to understand which (if at all) sacrifices are needed for that. I don't understand how we can just remove the nodejs dependency. That means, we don't use package.json, npm, etc. > > > > In the existing code, gulp helps in 1).compiling TS to JS 2) create > > sourcemaps 3) Combine JS files 4) Minify them 5) Copy to relevant path 6) > > Inject the script tags in html, and lot more. > > > > Make can do as well (in fact, I'm trying to find if make can't do > something"). > I did't say, make can't do this. I said, it would be more natural with Gulp. > > > > > The above can be achieved using Makefile, but i don't think its going to > be > > as straight forward as using gulp(or any nodejs based build system). > > > > Example, gulp: > =========== > gulp.task('sass', function () { > return gulp.src(path.styles) > .pipe($.sourcemaps.init()) > .pipe(sass()) > .pipe($.concat('tendrl.css')) > .pipe($.sourcemaps.write('.')) > .pipe(gulp.dest(path.dist + '/css')); > > Same thing, make: > ============= > sass: ${SASS} > cat $^ | sass --sourcemap compressed > ${DIST}/css/tendrl.css > > To me, second one is much more straight forward and more versatile. > It's leaner and faster as well, as it's executing the commands directly > instead of being wrapped within node-modules that does the proxying. > This is very basic example. Considering the following would be more appropriate, browserify({ entries: './app/app.js', debug: true }) .bundle() .pipe(source('app.js')) .pipe(buffer()) .pipe(ngAnnotate()) .pipe(gulpif(argv.prod, uglify())) .pipe(gulp.dest(path.dist + '/scripts')); > > > > > IMO, using Makfile for all kind of build tasks are not the way to go. > > Instead you can move from gulp+browserify to webpack and npm scripts if > you > > think gulp is verbose and complex. Then use the Makefile for packaging > and > > release. > > > Gulp isn't complex, and switching between gulp/grunt/npm-script is > the complete anti-theory of my proposal, unless we're using only > one solution. And make is years proven and well supported tool > to achieve the same thing - do we need these alternatives? > > In simpler words, what do we miss out if we move to `make` only? > There is no question about the capability of 'make', its about using a more approriate tool. > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From rkanade at redhat.com Mon Nov 7 12:50:42 2016 From: rkanade at redhat.com (Rohan Kanade) Date: Mon, 7 Nov 2016 18:20:42 +0530 Subject: [Tendrl-devel] Tendrl mid-cycle Hack Days Proposal (8, 9, 10 Nov 2016, Bangalore, India) In-Reply-To: References: Message-ID: Hack Days Document: https://docs.google.com/document/d/19Pp-ceUVmdwtMieZIRH08F1iSsB_8q8ULbfwXB6HOt0 Update with your specific agenda On Wed, Oct 26, 2016 at 12:19 PM, Rohan Kanade wrote: > Ken, > > https://github.com/Tendrl/ceph_bridge/blob/master/doc/ > source/installation.rst > > You can find similar doc for other tendrl repos. > > On Tue, Oct 25, 2016 at 9:28 PM, Ken Dreyer wrote: > >> On Mon, Oct 24, 2016 at 8:48 PM, Rohan Kanade wrote: >> > with Tendrl components installed and running. >> >> Would you please point me at the step-by-step instructions for how to >> do this? (Install all Tendrl components and get them running) >> >> - Ken >> >> _______________________________________________ >> Tendrl-devel mailing list >> Tendrl-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/tendrl-devel >> > > From rkanade at redhat.com Tue Nov 8 12:19:46 2016 From: rkanade at redhat.com (Rohan Kanade) Date: Tue, 8 Nov 2016 17:49:46 +0530 Subject: [Tendrl-devel] Tendrl 1 to be released on 14 Nov 2016 Message-ID: Tendrl 1 upstream will be released on 14 Nov 2016. This initial release will provide below features - Tendrl API first cut with documentation - Import of Ceph and Gluster clusters under Tendrl - Logging and API jobs framework for Tendrl components - Node and Cluster wide metrics - Demo Create via API: Pools, Bricks, Volumes - List views - Cluster (Ceph/Gluster), Node, Pools, File Shares Regards, Rohan Kanade From mkudlej at redhat.com Tue Nov 8 12:28:12 2016 From: mkudlej at redhat.com (Martin Kudlej) Date: Tue, 8 Nov 2016 13:28:12 +0100 Subject: [Tendrl-devel] Tendrl 1 to be released on 14 Nov 2016 In-Reply-To: References: Message-ID: <6a1a52ac-e082-902c-63bf-b721a1ba3ef8@redhat.com> Hi Rohan, all, On 11/08/2016 01:19 PM, Rohan Kanade wrote: > Tendrl 1 upstream will be released on 14 Nov 2016. This initial release > will provide below features > > - Tendrl API first cut with documentation > - Import of Ceph and Gluster clusters under Tendrl > - Logging and API jobs framework for Tendrl components > - Node and Cluster wide metrics > - Demo Create via API: Pools, Bricks, Volumes > - List views - Cluster (Ceph/Gluster), Node, Pools, File Shares could you please identify all Jira tasks which will be included in this release? So we can discuss about DoD for those tasks. I think there can be release only if DoD is valid for all tasks which will be included in Tendrl 1. Thank you! -- Best Regards, Martin Kudlej. RHSC/USM Senior Quality Assurance Engineer Red Hat Czech s.r.o. Phone: +420 532 294 155 E-mail:mkudlej at redhat.com IRC: mkudlej at #brno, #gluster, #storage-qa, #rhs, #rh-ceph, #usm-meeting @ redhat #tendrl-devel @ freenode From mrugesh at brainfunked.org Tue Nov 8 12:39:26 2016 From: mrugesh at brainfunked.org (Mrugesh Karnik) Date: Tue, 8 Nov 2016 18:09:26 +0530 Subject: [Tendrl-devel] Provisioning Integration for Ceph and Gluster In-Reply-To: References: Message-ID: On 5 November 2016 at 01:51, Alfredo Deza wrote: > > On Wed, Nov 2, 2016 at 2:29 AM, Nishanth Thomas wrote: > > On Mon, Oct 31, 2016 at 7:38 PM, Alfredo Deza wrote: > > > >> On Wed, Oct 26, 2016 at 11:45 PM, Nishanth Thomas > >> wrote: > >> > Hi Alfredo, > >> > > >> > Here is the first cut: > >> > > > We estimate that delivering all of the BZs isn't going to be possible > by the end of November. Description on effort per BZ inline. Where would you put the estimate at, if not November end? -- Mrugesh From fbalak at redhat.com Wed Nov 9 08:23:06 2016 From: fbalak at redhat.com (Filip Balak) Date: Wed, 9 Nov 2016 03:23:06 -0500 (EST) Subject: [Tendrl-devel] Gluster volume creation via API In-Reply-To: <2053528418.6742342.1478679118800.JavaMail.zimbra@redhat.com> Message-ID: <561828347.6742657.1478679786226.JavaMail.zimbra@redhat.com> Hi Nishanth, I tried to create volume via API. I installed Tendrl components with my playbook on Centos [1]. On gluster nodes I manually created bricks and connected them, after that I started node_agent and gluster_bridge. On server node I ran command bundle exec shotgun in Tendrl folder. When I tried to create gluster volume via API according to [2]. It returned: {"job_id":"08e4bf00-df25-4318-884c-e573e2aa1718","status":"processing","sds_nvr":"gluster-3.8.3","action":"create","object_type":"volume"} but when I checked gluster volumes with command: gluster volume status all, it didn't find any volumes. Is this expected behavior right now? If not, could you please provide me some advice how can I correctly create gluster volume via API? Many thanks, Filip Balak [1] https://github.com/Tendrl/usmqe-setup/tree/tendrl-ansible [2] https://github.com/Tendrl/documentation/blob/master/api/gluster-create-volume-example.adoc From mbukatov at redhat.com Wed Nov 9 14:37:12 2016 From: mbukatov at redhat.com (Martin Bukatovic) Date: Wed, 9 Nov 2016 15:37:12 +0100 Subject: [Tendrl-devel] tendrl aplha release Message-ID: Dear all, I have noticed[1] that we are going to have so called "alpha relese": > Confirm Tendrl 1 on Nov 14? Yes but concerns about getting to DoD > Tendrl 1 will be an alpha/developer release but does not cover > DoD just code completion and consumable. > Code ready on Monday but packages will come a few days. Which I would read as: 1) we have shifted proper upstream release into not yet specified point in the near future 2) this will not be reflected in status of the JIRA tasks in any way 3) QE team is not involved in this so called alpha release 4) we are going to have first serious builds/packages next week I have no problem with that as long as we clearly communicate that the original schedule is not met and all the planned features still needs to be finished and released later. This is hardly surprising for the qe team, who suggested that the initial schedule is overly optimistic not taking into account huge overhead of communication with other teams involved - such as ceph, gluster, blivet, freeipa... - which takes lot of time and effort and is crucial for the success of the project. So at this point, I see no problem with not meeting the original schedule as long as we are making good progress while moving into right direction. [1] https://docs.google.com/document/d/19Pp-ceUVmdwtMieZIRH08F1iSsB_8q8ULbfwXB6HOt0/edit -- Martin Bukatovic USM QE team From fbalak at redhat.com Wed Nov 9 15:57:35 2016 From: fbalak at redhat.com (Filip Balak) Date: Wed, 9 Nov 2016 10:57:35 -0500 (EST) Subject: [Tendrl-devel] Ansible playbook for deploying Tendrl API In-Reply-To: <1468593920.6807325.1478706999828.JavaMail.zimbra@redhat.com> Message-ID: <1548671726.6807394.1478707055680.JavaMail.zimbra@redhat.com> Hi all, I am trying to put together ansible playbook with roles to deploy Tendrl API, so I can test gluster volume creation. Please check it out [1] and review it. Best Regards, Filip Balak [1] https://github.com/Tendrl/usmqe-setup/tree/tendrl-ansible From kdreyer at redhat.com Wed Nov 9 17:41:15 2016 From: kdreyer at redhat.com (Ken Dreyer) Date: Wed, 9 Nov 2016 10:41:15 -0700 Subject: [Tendrl-devel] tendrl aplha release In-Reply-To: References: Message-ID: In that document, I see "Proposal: Scratch build in Koji only, Jenkins repo" Can we please use https://copr.fedorainfracloud.org/ instead of scratch builds and constructing things by hand? Copr will build and host the RPMs for you. It uses mock (just like Koji) so there's no "by hand" RPM builds. It will also be easier for all developers (myself included) to collaborate going forward if we use a tool that is purpose-built for this. - Ken On Wed, Nov 9, 2016 at 7:37 AM, Martin Bukatovic wrote: > Dear all, > > I have noticed[1] that we are going to have so called "alpha relese": > >> Confirm Tendrl 1 on Nov 14? Yes but concerns about getting to DoD >> Tendrl 1 will be an alpha/developer release but does not cover >> DoD just code completion and consumable. >> Code ready on Monday but packages will come a few days. > > Which I would read as: > > 1) we have shifted proper upstream release into not yet specified > point in the near future > 2) this will not be reflected in status of the JIRA tasks in any way > 3) QE team is not involved in this so called alpha release > 4) we are going to have first serious builds/packages next week > > I have no problem with that as long as we clearly communicate > that the original schedule is not met and all the planned features still > needs to be finished and released later. > > This is hardly surprising for the qe team, who suggested that the > initial schedule is overly optimistic not taking into account huge > overhead of communication with other teams involved - such as ceph, > gluster, blivet, freeipa... - which takes lot of time and effort > and is crucial for the success of the project. > > So at this point, I see no problem with not meeting the > original schedule as long as we are making good progress > while moving into right direction. > > [1] > https://docs.google.com/document/d/19Pp-ceUVmdwtMieZIRH08F1iSsB_8q8ULbfwXB6HOt0/edit > > -- > Martin Bukatovic > USM QE team > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From kdreyer at redhat.com Wed Nov 9 17:41:43 2016 From: kdreyer at redhat.com (Ken Dreyer) Date: Wed, 9 Nov 2016 10:41:43 -0700 Subject: [Tendrl-devel] CentOS Storage SIG meeting on Friday 28-Oct-2016 at 13:00 UTC In-Reply-To: References: <20161026185118.GD2936@ndevos-x240.usersys.redhat.com> <20161027165114.GG2936@ndevos-x240.usersys.redhat.com> Message-ID: On Fri, Nov 4, 2016 at 12:39 AM, Nishanth Thomas wrote: > On Wed, Nov 2, 2016 at 9:38 PM, Ken Dreyer wrote: >> We could use a separate Copr for Tendrl, and add all developers' FAS >> usernames to have permissions to add builds to it. Nishanth, want to >> create that? You can do it at https://copr.fedorainfracloud.org/coprs/ > > > Thanks Ken. Its a good option until the fedora packages are ready. I will > take care of this. Hi Nishanth, What's the URL to your copr? - Ken From japplewh at redhat.com Wed Nov 9 18:39:49 2016 From: japplewh at redhat.com (Jeff Applewhite) Date: Wed, 9 Nov 2016 13:39:49 -0500 Subject: [Tendrl-devel] tendrl aplha release In-Reply-To: References: Message-ID: Just to clarify - I think perhaps you missed parts of the meeting On Wed, Nov 9, 2016 at 9:37 AM, Martin Bukatovic wrote: > Dear all, > > I have noticed[1] that we are going to have so called "alpha relese": > > > Confirm Tendrl 1 on Nov 14? Yes but concerns about getting to DoD > > Tendrl 1 will be an alpha/developer release but does not cover > > DoD just code completion and consumable. > > Code ready on Monday but packages will come a few days. > > Which I would read as: > > 1) we have shifted proper upstream release into not yet specified > point in the near future > ? yes - we need QE to test the developer build to call it a release. Let's also be mindful that this is the first release - we will build our cadence - the packaging and related work and decisions involved are not trivial - let's not kid ourselves. I think Development committed to a Nov 14 delivery of packages. How long will QE need? Do you have APIs to develop against? ? > 2) this will not be reflected in status of the JIRA tasks in any way > ?Not true - everyone was asked to update their Jira tickets - during the hackathon they are focused on development and Github tickets for now - Jira does need to get in sync at the end of the hackathon ? > 3) QE team is not involved in this so called alpha release > ? refer to #1! Our definition of done requires testing!!! We are as committed to this as ever. ? > 4) we are going to have first serious builds/packages next week > > I have no problem with that as long as we clearly communicate > that the original schedule is not met and all the planned features still > needs to be finished and released later. > > This is hardly surprising for the qe team, who suggested that the > initial schedule is overly optimistic not taking into account huge > overhead of communication with other teams involved - such as ceph, > gluster, blivet, freeipa... - which takes lot of time and effort > and is crucial for the success of the project. > So at this point, I see no problem with not meeting the > original schedule as long as we are making good progress > while moving into right direction. > ?yes we are? ?making good progress - I hope to see the testing/automation/CI around the first release not be a blocker - in order for that to happen we need a cross disciplinary approach to testing and good advance notice of APIs from Dev > > [1] > https://docs.google.com/document/d/19Pp-ceUVmdwtMieZIRH08F1iSsB_ > 8q8ULbfwXB6HOt0/edit > > -- > Martin Bukatovic > USM QE team > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > -- Jeff Applewhite Principal Product Manager From nthomas at redhat.com Thu Nov 10 05:59:07 2016 From: nthomas at redhat.com (Nishanth Thomas) Date: Thu, 10 Nov 2016 11:29:07 +0530 Subject: [Tendrl-devel] CentOS Storage SIG meeting on Friday 28-Oct-2016 at 13:00 UTC In-Reply-To: References: <20161026185118.GD2936@ndevos-x240.usersys.redhat.com> <20161027165114.GG2936@ndevos-x240.usersys.redhat.com> Message-ID: On Wed, Nov 9, 2016 at 11:11 PM, Ken Dreyer wrote: > On Fri, Nov 4, 2016 at 12:39 AM, Nishanth Thomas > wrote: > > On Wed, Nov 2, 2016 at 9:38 PM, Ken Dreyer wrote: > >> We could use a separate Copr for Tendrl, and add all developers' FAS > >> usernames to have permissions to add builds to it. Nishanth, want to > >> create that? You can do it at https://copr.fedorainfracloud.org/coprs/ > > > > > > Thanks Ken. Its a good option until the fedora packages are ready. I will > > take care of this. > > Hi Nishanth, > > What's the URL to your copr? > https://copr.fedorainfracloud.org/coprs/nthomas75/tendrl/ I need to talk to you about this. I will set-up a bluejeans sometime today for the same > > - Ken > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From sankarshan.mukhopadhyay at gmail.com Thu Nov 10 06:12:55 2016 From: sankarshan.mukhopadhyay at gmail.com (Sankarshan Mukhopadhyay) Date: Thu, 10 Nov 2016 11:42:55 +0530 Subject: [Tendrl-devel] tendrl aplha release In-Reply-To: References: Message-ID: On Thu, Nov 10, 2016 at 12:09 AM, Jeff Applewhite wrote: > Just to clarify - I think perhaps you missed parts of the meeting > > On Wed, Nov 9, 2016 at 9:37 AM, Martin Bukatovic > wrote: > >> Dear all, >> >> I have noticed[1] that we are going to have so called "alpha relese": >> >> > Confirm Tendrl 1 on Nov 14? Yes but concerns about getting to DoD >> > Tendrl 1 will be an alpha/developer release but does not cover >> > DoD just code completion and consumable. >> > Code ready on Monday but packages will come a few days. >> >> Which I would read as: >> >> 1) we have shifted proper upstream release into not yet specified >> point in the near future >> > > yes - we need QE to test the developer build to call it a release. Let's > also be mindful that this is the first release - we will build our cadence > - the packaging and related work and decisions involved are not trivial - > let's not kid ourselves. I think Development committed to a Nov 14 delivery > of packages. How long will QE need? Do you have APIs to develop against? > I have mentioned this previously (in another thread) and I would like to reiterate on the topic. The 'definition of done' is a at this point a guidance on how across the teams involved we can be certain of completing everything that is required to ensure that a build is functional. The Definition of Done requires that a build needs to be tested. And Tendrl1 is as good a time as any to sort out the impediments to testing of builds. What are the list of things that we still remain to be completed before testing can start? Have we made progress in addressing those open items? The downside to have a late start to the testing process is that development time from future (Tendrl2 and Tendrl3) need to be carved out to address the Tendrl1 topics which come about. That by itself is not a risk, but would not be a nice thing to do. > > >> 2) this will not be reflected in status of the JIRA tasks in any way >> This has me puzzled. > > Not true - everyone was asked to update their Jira tickets - during the > hackathon they are focused on development and Github tickets for now - Jira > does need to get in sync at the end of the hackathon > This is the correct expectation. > >> 3) QE team is not involved in this so called alpha release >> > I am not sure why this hardline stance is being put forward. Are there constraints that make it impossible for QE to start and complete testing? If yes, we should work towards resolving those. Even if a release is tagged as "alpha", if a group with testing expertise arbitrarily decides to be hands-off - that is a cause for concern. > refer to #1! Our definition of done requires testing!!! We are as committed > to this as ever. > > I could not agree more. I'd rather want to understand what are the real and tangible blockers to start of testing and address those. Unless we are in the discipline of development and testing - we will not be able to land continuous and iterative builds which are of any value to a potential user. > >> 4) we are going to have first serious builds/packages next week >> Do you have all the things required and prepped to be able to test these builds? >> I have no problem with that as long as we clearly communicate >> that the original schedule is not met and all the planned features still >> needs to be finished and released later. >> >> This is hardly surprising for the qe team, who suggested that the >> initial schedule is overly optimistic not taking into account huge >> overhead of communication with other teams involved - such as ceph, >> gluster, blivet, freeipa... - which takes lot of time and effort >> and is crucial for the success of the project. > Ok. Here's the reality - the "storage management project" will have to coordinate moving pieces with storage systems, file-systems and a host of other projects. This is not *overhead* - this is how projects work. Let us be clear, even Ceph and Gluster have enough interlocking aspects with different projects that make it a natural part of a software development cycle. All projects have fluid release plans and priorities - adjusting them for a win-win requires conversations and in initial stages they require more than usual share of such. > >> So at this point, I see no problem with not meeting the >> original schedule as long as we are making good progress >> while moving into right direction. >> > > yes we are > > making good progress - I hope to see the testing/automation/CI around the > first release not be a blocker - in order for that to happen we need a > cross disciplinary approach to testing and good advance notice of APIs from > Dev > >> >> [1] >> https://docs.google.com/document/d/19Pp-ceUVmdwtMieZIRH08F1iSsB_ >> 8q8ULbfwXB6HOt0/edit -- sankarshan mukhopadhyay From mbukatov at redhat.com Thu Nov 10 08:35:27 2016 From: mbukatov at redhat.com (Martin Bukatovic) Date: Thu, 10 Nov 2016 09:35:27 +0100 Subject: [Tendrl-devel] tendrl aplha release In-Reply-To: References: Message-ID: <3bb4835f-f66b-1b66-c16a-ae8dfa2ed41d@redhat.com> On 11/09/2016 06:41 PM, Ken Dreyer wrote: > In that document, I see "Proposal: Scratch build in Koji only, Jenkins repo" > > Can we please use https://copr.fedorainfracloud.org/ instead of > scratch builds and constructing things by hand? Copr will build and > host the RPMs for you. It uses mock (just like Koji) so there's no "by > hand" RPM builds. It will also be easier for all developers (myself > included) to collaborate going forward if we use a tool that is > purpose-built for this. I agree here. Using copr at first would be easier for developers and it would make the consumption of the builds easier for anybody (qe team included) at the same time. -- Martin Bukatovic USM QE team From mbukatov at redhat.com Thu Nov 10 10:01:52 2016 From: mbukatov at redhat.com (Martin Bukatovic) Date: Thu, 10 Nov 2016 11:01:52 +0100 Subject: [Tendrl-devel] tendrl aplha release In-Reply-To: References: Message-ID: On 11/09/2016 07:39 PM, Jeff Applewhite wrote: > Just to clarify - I think perhaps you missed parts of the meeting Thanks for the reply, I think that we mostly agree with each other, I just wanted to make sure we don't kid themselves and not call something finished when it was neither integrated, not installed in a expected production setup nor tested. Any self respecting QE team is very sensitive to this kind of things and we just can't allow to lower quality requirements now. Getting the project rolling in the right way would pay heavily in the end. And don't get me wrong, I don't suggest that we are either doing it or planning to do so. We have a good definition of done we all agreed on and I just wanted to make sure that we are all on the same page here and put the alpha plan into the right perspective. >> Which I would read as: >> >> 1) we have shifted proper upstream release into not yet specified >> point in the near future >> > ? > yes - we need QE to test the developer build to call it a release. Let's > also be mindful that this is the first release - we will build our cadence > - the packaging and related work and decisions involved are not trivial - > let's not kid ourselves. I think Development committed to a Nov 14 delivery > of packages. Yes, the packages are important. With those packages, QE team can finish the most important QE task at this point: automation of fully production like test machines deployment for integration testing. This is necessary because the main work of the QE team here is the integration testing. Tendlr has lot of components and uses/talks to lot of others. For this reason, we need to make sure that we understand which component should be installed on which server role, which server roles needs to be deployed on dedicated machines and so on. Then, when we build an automation tests based on this infrastructure, only then we meet the definition of done we all agreed, which would allow us to move JIRA tasks to completed and to call it a proper release. > How long will QE need? Do you have APIs to develop against? So far, the only fully documented API is gluster volume create use case. Wrt to the implementation, it's hard to say as the related code is being worked on heavily during current hack days. Which is good. But to answer your question: I have no idea at this point, we are likely to discuss lot of details with developers related to the installation (as described above) and usage. Filip started working on the create volume tests based on the current information already, but it would need the be tweaked when the fully integrated tendrl setup is ready. >> 2) this will not be reflected in status of the JIRA tasks in any way >> > > ?Not true - everyone was asked to update their Jira tickets - during the > hackathon they are focused on development and Github tickets for now - Jira > does need to get in sync at the end of the hackathon I agree with this. I mean something different: to make alpha release happen on Monday, we would be releasing it without all the JIRA tasks to be in done/accepted state (based on the definition of done). I just wanted to make sure we don't trick themselves into thinking that those tasks are done. That said, having first rpm packages is an important milestone in a work towards the first upstream release, I have no problem calling it alpha release. >> 3) QE team is not involved in this so called alpha release >> > ? > refer to #1! Our definition of done requires testing!!! We are as committed > to this as ever. I agree here as well that our definition of done requires testing - on a task level. And a release, to deserve it's name, requires integration testing. And that's exactly the reason we can't do proper release on Monday, as the tendrl stack hasn't been fully integrated yet. But as I said, I think it's a great idea to call the first rpm build milestone as and alpha release. And since we would start the proper testing with those rpm packages, we don't do any actual QE work for this aplha release - quite the contrary, we start consuming the alpha builds. I just wanted to state this in a clear way. We organized meeting with Mrugesh on this Friday to discuss the current state of integration for us to be able to install the tendl components properly for the first time. > I hope to see the testing/automation/CI around the > first release not be a blocker - in order for that to happen we need a > cross disciplinary approach to testing and good advance notice of APIs from > Dev Here I disagree. The first release (I don't mean the alpha release) is indeed blocked by testing/automation/CI! This is one of the most important parts of the first release after all! How would we know that the stuff we release is worth of anything, if we don't do the proper testing, which is part of the definition of done? -- Martin Bukatovic USM QE team From julim at redhat.com Thu Nov 10 13:07:38 2016 From: julim at redhat.com (Ju Lim) Date: Thu, 10 Nov 2016 08:07:38 -0500 Subject: [Tendrl-devel] Tendrl Hackdays Day 3 recap/read-out Message-ID: Team: Just wanted to provide a quick summary/recap so far so folks can follow the progress so far as we get closer to releasing Tendrl1 (currently targeted for Monday, Nov 14 2016). For the hackdays, the team has accomplished the following (these are just highlights): - Completed pending reviews/changes for PRs for all components - Finalized on API ports for knowledge of external integrations like OSPD etc - Finalized the api job structure related to ?action? / ?flow? attribute - Finalized what extras to be passed as part of api job for pre/post run - Logging essentials completed - Completed validations for pre/post runs for volume management - Created the RPM specs for review - 90% unit test coverage - UI: repo restructured, old code shelved in ?legacy? branch, branding changes, documentation added, test framework added, cleanup of bitrot codes, header and navigation implementation are DONE. - Integration of UI/API with backend volume management - Worked on the UI and API required for the import gluster cluster flow, but they still need to be wired together What's remaining / outstanding: - Wiring the API and UI for the import gluster cluster flow - Import Ceph cluster - Documentation needed for the installation - Documentation CRUD operations - CI - pending RPM spec reviews, getting builds into Centos, etc. (so QE can start with integration testing) - Getting JIRA updated to reflect all the work completed so far - i.e. linking to GitHub issues, updating status, assigning stories/tasks to QE accordingly, etc. - Getting to Definition of Done (DoD) for all of the work items related to the Tendrl 1 release We will have a demo tomorrow to showcase the work completed during the Hackdays. If I missed anything noteworthy in this recap/read-out, please feel free to add / edit this thread. Thank you, Ju From kdreyer at redhat.com Thu Nov 10 14:44:53 2016 From: kdreyer at redhat.com (Ken Dreyer) Date: Thu, 10 Nov 2016 07:44:53 -0700 Subject: [Tendrl-devel] copr (was: CentOS Storage SIG meeting on Friday 28-Oct-2016 at 13:00 UTC) Message-ID: On Wed, Nov 9, 2016 at 10:59 PM, Nishanth Thomas wrote: > On Wed, Nov 9, 2016 at 11:11 PM, Ken Dreyer wrote: >> Hi Nishanth, >> >> What's the URL to your copr? >> > > https://copr.fedorainfracloud.org/coprs/nthomas75/tendrl/ > > I need to talk to you about this. I will set-up a bluejeans sometime today > for the same Thanks. I've requested access (FAS login: ktdreyer) What is a really simple package we can put there first, to ensure it's working? (Some small Python dependency that's not already in CentOS core?) For the Copr settings for each build, I prefer to set "Mock build from a SCM repository", because then you can build the RPMs directly from GitHub, without having to generate SRPMs and upload them. See https://copr.fedorainfracloud.org/coprs/ktdreyer/downstream-cherry-picker/package/downstream-cherry-picker/ for an example package that uses these settings. - Ken From julim at redhat.com Fri Nov 11 14:20:15 2016 From: julim at redhat.com (Ju Lim) Date: Fri, 11 Nov 2016 09:20:15 -0500 Subject: [Tendrl-devel] Import Gluster Cluster Walk-thru Demo (Fri, 11 Nov 2016) Message-ID: Team: Please find below a link to today's review/demo that shows and talks about the progress made to-date on the Import Gluster Cluster flow: https://bluejeans.com/s/1HYTr/ Demo/review highlights: - provides a walk-through of how the Import Gluster Cluster flow works from uploading the definitions (YML) file all the way through execution of the flow - firing up the API query - UI showing a basic import cluster form, but it's not yet firing up the API at the moment Work still remaining (for Import Cluster): - Complete wiring up the Import Gluster Cluster (API + UI) -- team is still debugging issues with individual atoms - Doing the Ceph cluster import (which should be similar though it will require an additional atom to handle Mons) For more details about what's outstanding, please refer the read-out/recap from Day 3 . Thank you, Ju From adeza at redhat.com Mon Nov 14 15:17:11 2016 From: adeza at redhat.com (Alfredo Deza) Date: Mon, 14 Nov 2016 10:17:11 -0500 Subject: [Tendrl-devel] Provisioning Integration for Ceph and Gluster In-Reply-To: References: Message-ID: On Tue, Nov 8, 2016 at 7:39 AM, Mrugesh Karnik wrote: > On 5 November 2016 at 01:51, Alfredo Deza wrote: >> >> On Wed, Nov 2, 2016 at 2:29 AM, Nishanth Thomas > wrote: >> > On Mon, Oct 31, 2016 at 7:38 PM, Alfredo Deza wrote: >> > >> >> On Wed, Oct 26, 2016 at 11:45 PM, Nishanth Thomas >> >> wrote: >> >> > Hi Alfredo, >> >> > >> >> > Here is the first cut: >> >> > >> >> We estimate that delivering all of the BZs isn't going to be possible >> by the end of November. Description on effort per BZ inline. > > Where would you put the estimate at, if not November end? To complete all BZs we are looking at roughly 1.5 months of Andrew and I working on them. At this point we don't think we can provide undivided attention to complete that effort. Since we are already two weeks into November we are already looking at early to mid December to complete this BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1309415 > > -- > Mrugesh > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From rkanade at redhat.com Mon Nov 14 15:40:51 2016 From: rkanade at redhat.com (Rohan Kanade) Date: Mon, 14 Nov 2016 21:10:51 +0530 Subject: [Tendrl-devel] Release Update : Tendrl 1 Message-ID: Tendrl 1 is being developer tested and is going through more bug fixes. The first release should happen on satisfactory developer testing Revised Release ETA: 15 Nov 2016 Regards, Rohan Kanade From johfulto at redhat.com Tue Nov 15 15:40:14 2016 From: johfulto at redhat.com (John Fulton) Date: Tue, 15 Nov 2016 10:40:14 -0500 Subject: [Tendrl-devel] compute resources necessary for tendrl Message-ID: Do we have the resource requirements for Tendrl documented? I am looking into integrating Tendrl with TripleO [1] and will need to get a small Tendrl server running in TripleO CI. Because TripleO CI is resource constrained I'd like to get an idea how much RAM/CPU/Disk I can get away for running Tendrl from the list so I can provide that detail to the TripleO team. Thanks, John [1] https://review.openstack.org/#/c/387631/ From kdreyer at redhat.com Tue Nov 15 16:28:02 2016 From: kdreyer at redhat.com (Ken Dreyer) Date: Tue, 15 Nov 2016 09:28:02 -0700 Subject: [Tendrl-devel] what are the Ubuntu agent packages? In-Reply-To: References: Message-ID: Hi Nishanth, what are the Tendrl agent packages that we'll need to create for Ubuntu Ceph clusters? On Wed, Nov 2, 2016 at 10:14 AM, Ken Dreyer wrote: > From what I understand, Tendrl has some "agent" packages that must run > on each Ceph node? > > What are the Tendrl agent packages that we will need to create for > Ubuntu Ceph clusters? > > - Ken From shtripat at redhat.com Tue Nov 15 18:10:43 2016 From: shtripat at redhat.com (Shubhendu Tripathi) Date: Tue, 15 Nov 2016 23:40:43 +0530 Subject: [Tendrl-devel] what are the Ubuntu agent packages? In-Reply-To: References: Message-ID: <10c900c9-d54b-09c9-0288-7d604dfac06e@redhat.com> I feel it would be below list - tendrl-node-agent - tendrl-ceph-integration - tendrl-common @Nishanth, correct me if names are not correct :) Regards, Shubhendu On 11/15/2016 09:58 PM, Ken Dreyer wrote: > Hi Nishanth, what are the Tendrl agent packages that we'll need to > create for Ubuntu Ceph clusters? > > On Wed, Nov 2, 2016 at 10:14 AM, Ken Dreyer wrote: >> From what I understand, Tendrl has some "agent" packages that must run >> on each Ceph node? >> >> What are the Tendrl agent packages that we will need to create for >> Ubuntu Ceph clusters? >> >> - Ken > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From kdreyer at redhat.com Tue Nov 15 18:21:10 2016 From: kdreyer at redhat.com (Ken Dreyer) Date: Tue, 15 Nov 2016 11:21:10 -0700 Subject: [Tendrl-devel] what are the Ubuntu agent packages? In-Reply-To: <10c900c9-d54b-09c9-0288-7d604dfac06e@redhat.com> References: <10c900c9-d54b-09c9-0288-7d604dfac06e@redhat.com> Message-ID: Thanks Shubhendu, I will look into packaging those for Xenial and host them at https://launchpad.net/~kdreyer-redhat/+archive/ubuntu/tendrl. - Ken On Tue, Nov 15, 2016 at 11:10 AM, Shubhendu Tripathi wrote: > I feel it would be below list > - tendrl-node-agent > - tendrl-ceph-integration > - tendrl-common > > @Nishanth, correct me if names are not correct :) > > Regards, > Shubhendu > > > > On 11/15/2016 09:58 PM, Ken Dreyer wrote: >> >> Hi Nishanth, what are the Tendrl agent packages that we'll need to >> create for Ubuntu Ceph clusters? >> >> On Wed, Nov 2, 2016 at 10:14 AM, Ken Dreyer wrote: >>> >>> From what I understand, Tendrl has some "agent" packages that must run >>> on each Ceph node? >>> >>> What are the Tendrl agent packages that we will need to create for >>> Ubuntu Ceph clusters? >>> >>> - Ken >> >> _______________________________________________ >> Tendrl-devel mailing list >> Tendrl-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/tendrl-devel > > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From mbukatov at redhat.com Wed Nov 16 12:05:44 2016 From: mbukatov at redhat.com (Martin Bukatovic) Date: Wed, 16 Nov 2016 13:05:44 +0100 Subject: [Tendrl-devel] Release Update : Tendrl 1 In-Reply-To: References: Message-ID: On 11/14/2016 04:40 PM, Rohan Kanade wrote: > Tendrl 1 is being developer tested and is going through more bug fixes. The > first release should happen on satisfactory developer testing I guess you have the "alpha release" in mind - which is basically a first important milestone, which would contain rpm builds and full description of a production setup. I'm pointing this out because the first release requires clear definition in tendrl jira (listing all tasks which are part of the release and making sure all of those are in accepted state) and I would not like others to confuse those two releases. While this may seem like a nitpicking, I see a value in listing facts in a clear way. -- Martin Bukatovic USM QE team From japplewh at redhat.com Wed Nov 16 19:41:52 2016 From: japplewh at redhat.com (Jeff Applewhite) Date: Wed, 16 Nov 2016 14:41:52 -0500 Subject: [Tendrl-devel] sprint 005 planning Message-ID: Hi All The planned items for sprint five are listed on the sprint planning page here: https://tendrl.atlassian.net/wiki/display/TEN/Sprints Please review and add details as appropriate. Mrugesh/Nishanth - please re-assign as appropriate. -- Jeff Applewhite Principal Product Manager From rkanade at redhat.com Thu Nov 17 06:38:51 2016 From: rkanade at redhat.com (Rohan Kanade) Date: Thu, 17 Nov 2016 12:08:51 +0530 Subject: [Tendrl-devel] [Release] Tendrl v1.0 Message-ID: We are pleased to announce the release of Tendrl v1.0. The first release brings capabilities for importing Ceph and Gluster clusters and primary CRUD operations on cluster objects like Volumes, Peers, Pools etc. Please refer to individual Tendrl components for more details about this release: Tendrl-common: https://github.com/Tendrl/common/releases/tag/v1.0 Tendrl-node-agent: https://github.com/Tendrl/node_agent/releases/tag/v1.0 Tendrl-ceph-integration: https://github.com/Tendrl/ceph_integration/releases/tag/v1.0 Tendrl-gluster-integration: https://github.com/Tendrl/gluster_integration/releases/tag/v1.0 Thank you to all Tendrl contributors. Regards, Rohan Kanade From mrugesh at brainfunked.org Thu Nov 17 13:10:08 2016 From: mrugesh at brainfunked.org (Mrugesh Karnik) Date: Thu, 17 Nov 2016 18:40:08 +0530 Subject: [Tendrl-devel] Provisioning Integration for Ceph and Gluster In-Reply-To: References: Message-ID: On 14 November 2016 at 20:47, Alfredo Deza wrote: > On Tue, Nov 8, 2016 at 7:39 AM, Mrugesh Karnik wrote: > > On 5 November 2016 at 01:51, Alfredo Deza wrote: > >> We estimate that delivering all of the BZs isn't going to be possible > >> by the end of November. Description on effort per BZ inline. > > > > Where would you put the estimate at, if not November end? > > To complete all BZs we are looking at roughly 1.5 months of Andrew and > I working on them. > > At this point we don't think we can provide undivided attention to > complete that effort. Since we are already two weeks into November > we are already looking at early to mid December to complete this BZ: > > https://bugzilla.redhat.com/show_bug.cgi?id=1309415 The callbacks functionality isn't going to be required by Tendrl, polling is sufficient. Would it be possible for you to focus on the other issues, instead, and provide us with a more concrete timeframe for by when work on them could be completed? Thanks, Mrugesh From japplewh at redhat.com Thu Nov 17 13:40:01 2016 From: japplewh at redhat.com (Jeff Applewhite) Date: Thu, 17 Nov 2016 08:40:01 -0500 Subject: [Tendrl-devel] [Release] Tendrl v1.0 In-Reply-To: References: Message-ID: Congratulations! On Thursday, November 17, 2016, Rohan Kanade wrote: > We are pleased to announce the release of Tendrl v1.0. > > > The first release brings capabilities for importing Ceph and Gluster > clusters and primary CRUD operations on cluster objects like Volumes, > Peers, Pools etc. > > > Please refer to individual Tendrl components for more details about this > release: > > Tendrl-common: https://github.com/Tendrl/common/releases/tag/v1.0 > > Tendrl-node-agent: https://github.com/Tendrl/node_agent/releases/tag/v1.0 > > Tendrl-ceph-integration: > https://github.com/Tendrl/ceph_integration/releases/tag/v1.0 > > Tendrl-gluster-integration: > https://github.com/Tendrl/gluster_integration/releases/tag/v1.0 > > > Thank you to all Tendrl contributors. > > Regards, > Rohan Kanade > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > -- Jeff Applewhite Principal Product Manager From kdreyer at redhat.com Thu Nov 17 17:03:17 2016 From: kdreyer at redhat.com (Ken Dreyer) Date: Thu, 17 Nov 2016 10:03:17 -0700 Subject: [Tendrl-devel] [Release] Tendrl v1.0 In-Reply-To: References: Message-ID: Hi Rohan, I don't see packages in https://copr.fedorainfracloud.org/coprs/nthomas75/tendrl/ . When will the RPMs be published? - Ken On Wed, Nov 16, 2016 at 11:38 PM, Rohan Kanade wrote: > We are pleased to announce the release of Tendrl v1.0. > > > The first release brings capabilities for importing Ceph and Gluster > clusters and primary CRUD operations on cluster objects like Volumes, > Peers, Pools etc. > > > Please refer to individual Tendrl components for more details about this > release: > > Tendrl-common: https://github.com/Tendrl/common/releases/tag/v1.0 > > Tendrl-node-agent: https://github.com/Tendrl/node_agent/releases/tag/v1.0 > > Tendrl-ceph-integration: > https://github.com/Tendrl/ceph_integration/releases/tag/v1.0 > > Tendrl-gluster-integration: > https://github.com/Tendrl/gluster_integration/releases/tag/v1.0 > > > Thank you to all Tendrl contributors. > > Regards, > Rohan Kanade > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From anivargi at redhat.com Thu Nov 17 21:03:49 2016 From: anivargi at redhat.com (Anup Nivargi) Date: Fri, 18 Nov 2016 02:33:49 +0530 Subject: [Tendrl-devel] Tendrl API v1.0 upstream release Message-ID: Hi, Tendrl API v1.0 has been released upstream [1] The release includes the initial framework setup along with a few storage management API's like importing a Ceph cluster, importing a Gluster cluster, creating/deleting a Ceph pool and creating/deleting a Gluster volume. It also provides basic features like to list nodes, clusters, volumes, peers and pools. [1] https://github.com/Tendrl/tendrl-api/releases/tag/v1.0 -- Anup Nivargi From rwheeler at redhat.com Fri Nov 18 07:16:20 2016 From: rwheeler at redhat.com (Ric Wheeler) Date: Fri, 18 Nov 2016 09:16:20 +0200 Subject: [Tendrl-devel] [Release] Tendrl v1.0 In-Reply-To: References: Message-ID: Congratulations on hitting this milestone! Ric On Nov 17, 2016 08:38, "Rohan Kanade" wrote: > We are pleased to announce the release of Tendrl v1.0. > > > The first release brings capabilities for importing Ceph and Gluster > clusters and primary CRUD operations on cluster objects like Volumes, > Peers, Pools etc. > > > Please refer to individual Tendrl components for more details about this > release: > > Tendrl-common: https://github.com/Tendrl/common/releases/tag/v1.0 > > Tendrl-node-agent: https://github.com/Tendrl/node_agent/releases/tag/v1.0 > > Tendrl-ceph-integration: > https://github.com/Tendrl/ceph_integration/releases/tag/v1.0 > > Tendrl-gluster-integration: > https://github.com/Tendrl/gluster_integration/releases/tag/v1.0 > > > Thank you to all Tendrl contributors. > > Regards, > Rohan Kanade > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From anbabu at redhat.com Fri Nov 18 10:57:12 2016 From: anbabu at redhat.com (Anmol Babu) Date: Fri, 18 Nov 2016 05:57:12 -0500 (EST) Subject: [Tendrl-devel] Tendrl Flow Definitions In-Reply-To: <481945992.183509.1479460308809.JavaMail.zimbra@redhat.com> Message-ID: <885696456.190941.1479466632538.JavaMail.zimbra@redhat.com> Hi all, I have a suggestion for a very small enhancement in the current flow framework. Current working procedure: 1. Any service/agent defining and implementing a flow pushes the corresponding flow definitions to its specific definitions etcd directory 2. The Agent/service flow framework reads its specific definition files and does all validations and executes the flow. ex: 1. Node-agent pushes tendrl_definitions_node_agent to /tendrl_definitions_node_agent in etcd, gluster integration pushes tendrl_definitions_gluster_integration to /tendrl_definitions_gluster_integration and so on... 2. node-agent is a service that reads flow definitions from /tendrl_definitions_node_agent in etcd and gluster-integration is a process that reads /tendrl_definitions_gluster_integration from etcd and currently each of them replicates the flow definition parsing logic(probably its already currently thought through to move generic sections of flow yaml parsing to tendrl/common) Drawback: Any component requiring to define its own flows will then need to implement(currently followed)/extend(from a generic one in tendrl/common -- probably already thought of) the flow framework and this requires the component to be run as a service as the flow framework in each of these component will only and always read only its specific definition and whenever there is a job(intended for it) in the etcd queue. Suggestion: 1. Push all flow definitions to a path under a common root directory say for ex: /tendrl_definitons/ in etcd 2. Now, when there's a job in the queue, the flow framework will make use of either: i. A part of the full qualified package name in the run parameter of the job or ii. A separate field to indicate the definitions namespace(i.e, the name under the /tendrl_definitions/ where the definition can be obtained from) to read definitions from appropriate etcd directory. 3. And from there the already existing flow framework as usual will extract and execute the pre, post and the normal flow atoms. Relevance of the suggestion: 1. There is a central application called tendrl/performance_monitoring which performs the task of i. Statistics aggregation ii. Exposing time series db data iii. Loading monitoring specific initial flow definitions to etcd 2. There will be a node-side library that defines atoms and flows specific to monitoring. This is carved out from tendrl/performance_monitoring application(separate rpms). The flow definition(yaml) being specific to monitoring, cannot be part of flow definitions(yaml) of any other project i.e, node_agent or even the gluster-integration or ceph-integration because monitoring is intended to be completely outside core stack. Now, given this, consider the case of a monitoring flow to generate collectd configurations on the nodes. In such a case the idea is to employ a python script(wrapper around jinja2 templating) that is executed as a command by the flow (This is not still generic as internally the command is not just a mere jinja2 wrapper but contains some specifics of collectd). So that the flow is very generic(with very specific parameters). But exposing this as a flow to execute any normal command can turn out to be a breach of security and hence the command needs to be hard-coded in the flow's respective atom which makes it still monitoring specific and hence needs to be part of an application that is specific to monitoring. But, with the current flow framework, just so that flow is executed, the flow implementer needs to be a service. But, with the small change suggested above the implementing application can be a mere stateless library(like tendrl/common) and any and every flow execution can be streamlined to be executed by the node-agent by just a simple python import of the specific installed python source which can be provided by the library(specific tendrl project). Please consider this idea and provide your valuable suggestions/feedback. Thanks, Anmol From shtripat at redhat.com Fri Nov 18 11:43:10 2016 From: shtripat at redhat.com (Shubhendu Tripathi) Date: Fri, 18 Nov 2016 17:13:10 +0530 Subject: [Tendrl-devel] Tendrl Flow Definitions In-Reply-To: <885696456.190941.1479466632538.JavaMail.zimbra@redhat.com> References: <885696456.190941.1479466632538.JavaMail.zimbra@redhat.com> Message-ID: I agree and this is something which we discussed yesterday in detail. Request others to comment. Regards, Shubhendu On 11/18/2016 04:27 PM, Anmol Babu wrote: > Hi all, > > I have a suggestion for a very small enhancement in the current flow framework. > > Current working procedure: > > 1. Any service/agent defining and implementing a flow pushes the corresponding flow definitions to its specific definitions etcd directory > 2. The Agent/service flow framework reads its specific definition files and does all validations and executes the flow. > > ex: > 1. Node-agent pushes tendrl_definitions_node_agent to /tendrl_definitions_node_agent in etcd, gluster integration pushes tendrl_definitions_gluster_integration to /tendrl_definitions_gluster_integration and so on... > 2. node-agent is a service that reads flow definitions from /tendrl_definitions_node_agent in etcd and gluster-integration is a process that reads /tendrl_definitions_gluster_integration from etcd and currently each of them > replicates the flow definition parsing logic(probably its already currently thought through to move generic sections of flow yaml parsing to tendrl/common) > > Drawback: > > Any component requiring to define its own flows will then need to implement(currently followed)/extend(from a generic one in tendrl/common -- probably already thought of) the flow framework > and this requires the component to be run as a service as the flow framework in each of these component will only and always read only its specific definition and whenever there is a job(intended for it) > in the etcd queue. > > Suggestion: > > 1. Push all flow definitions to a path under a common root directory say for ex: /tendrl_definitons/ in etcd > 2. Now, when there's a job in the queue, the flow framework will make use of either: > i. A part of the full qualified package name in the run parameter of the job > or > ii. A separate field to indicate the definitions namespace(i.e, the name under the /tendrl_definitions/ where the definition can be obtained from) > to read definitions from appropriate etcd directory. > 3. And from there the already existing flow framework as usual will extract and execute the pre, post and the normal flow atoms. > > Relevance of the suggestion: > > 1. There is a central application called tendrl/performance_monitoring which performs the task of > i. Statistics aggregation > ii. Exposing time series db data > iii. Loading monitoring specific initial flow definitions to etcd > 2. There will be a node-side library that defines atoms and flows specific to monitoring. This is carved out from tendrl/performance_monitoring application(separate rpms). > The flow definition(yaml) being specific to monitoring, cannot be part of flow definitions(yaml) of any other project i.e, node_agent or even the gluster-integration or ceph-integration > because monitoring is intended to be completely outside core stack. Now, given this, consider the case of a monitoring flow to generate collectd configurations on the nodes. In such a case > the idea is to employ a python script(wrapper around jinja2 templating) that is executed as a command by the flow > (This is not still generic as internally the command is not just a mere jinja2 wrapper but contains some specifics of collectd). > So that the flow is very generic(with very specific parameters). But exposing > this as a flow to execute any normal command can turn out to be a breach of security and hence the command needs to be hard-coded in the flow's respective atom which makes it still monitoring specific > and hence needs to be part of an application that is specific to monitoring. But, with the current flow framework, just so that flow is executed, the flow implementer needs to be a service. > But, with the small change suggested above the implementing application can be a mere stateless library(like tendrl/common) and any and every flow execution can be streamlined to be executed by the node-agent > by just a simple python import of the specific installed python source which can be provided by the library(specific tendrl project). > > > Please consider this idea and provide your valuable suggestions/feedback. > > > Thanks, > Anmol From smohan at redhat.com Fri Nov 18 11:48:25 2016 From: smohan at redhat.com (Satish Mohan) Date: Fri, 18 Nov 2016 17:18:25 +0530 Subject: [Tendrl-devel] [Release] Tendrl v1.0 In-Reply-To: References: Message-ID: Congratulations! ~Satish On Thu, Nov 17, 2016 at 12:08 PM, Rohan Kanade wrote: > We are pleased to announce the release of Tendrl v1.0. > > > The first release brings capabilities for importing Ceph and Gluster > clusters and primary CRUD operations on cluster objects like Volumes, > Peers, Pools etc. > > > Please refer to individual Tendrl components for more details about this > release: > > Tendrl-common: https://github.com/Tendrl/common/releases/tag/v1.0 > > Tendrl-node-agent: https://github.com/Tendrl/node_agent/releases/tag/v1.0 > > Tendrl-ceph-integration: > https://github.com/Tendrl/ceph_integration/releases/tag/v1.0 > > Tendrl-gluster-integration: > https://github.com/Tendrl/gluster_integration/releases/tag/v1.0 > > > Thank you to all Tendrl contributors. > > Regards, > Rohan Kanade > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From flucifre at redhat.com Fri Nov 18 11:50:55 2016 From: flucifre at redhat.com (Federico Lucifredi) Date: Fri, 18 Nov 2016 06:50:55 -0500 Subject: [Tendrl-devel] [Release] Tendrl v1.0 In-Reply-To: References: Message-ID: <103652318247358768@unknownmsgid> Congratulations guys! Best-F > On Nov 18, 2016, at 6:48 AM, Satish Mohan wrote: > > Congratulations! > > ~Satish > >> On Thu, Nov 17, 2016 at 12:08 PM, Rohan Kanade wrote: >> >> We are pleased to announce the release of Tendrl v1.0. >> >> >> The first release brings capabilities for importing Ceph and Gluster >> clusters and primary CRUD operations on cluster objects like Volumes, >> Peers, Pools etc. >> >> >> Please refer to individual Tendrl components for more details about this >> release: >> >> Tendrl-common: https://github.com/Tendrl/common/releases/tag/v1.0 >> >> Tendrl-node-agent: https://github.com/Tendrl/node_agent/releases/tag/v1.0 >> >> Tendrl-ceph-integration: >> https://github.com/Tendrl/ceph_integration/releases/tag/v1.0 >> >> Tendrl-gluster-integration: >> https://github.com/Tendrl/gluster_integration/releases/tag/v1.0 >> >> >> Thank you to all Tendrl contributors. >> >> Regards, >> Rohan Kanade >> _______________________________________________ >> Tendrl-devel mailing list >> Tendrl-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/tendrl-devel >> > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From amukherj at redhat.com Mon Nov 21 07:13:02 2016 From: amukherj at redhat.com (Atin Mukherjee) Date: Mon, 21 Nov 2016 12:43:02 +0530 Subject: [Tendrl-devel] Regarding some additional data points in the state output In-Reply-To: <5e1609e4-cfc3-4dc3-f421-ac51f56b9d64@redhat.com> References: <5e1609e4-cfc3-4dc3-f421-ac51f56b9d64@redhat.com> Message-ID: +tendrl-devel On Mon, Nov 21, 2016 at 12:41 PM, Samikshan Bairagya wrote: > Hey Rohan, > > So the current get-state CLI misses volume specific options in its output. > Somehow I missed it while coming up with the implementation. This patch by > Atin is a fix for that: http://review.gluster.org/#/c/15889/1. The > following example shows how this patch would add these new data points and > how that would change the existing format: > > [Volumes] > Volume1.name: tv1 > Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 > Volume1.type: Distribute > > Volume1.rebalance.files: 0 > Volume1.rebalance.data: 0Bytes > [Volume1.options] > nfs.disable: on > performance.readdir-ahead: on > transport.address-family: inet > features.uss: on > > Volume2.name: tv2 > Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a > Volume2.type: Distribute > ......... > ......... > > So essentially there would be a new section for every volume that would > list the option names and corresponding values. Would adding this change > still keep the get-state output parseable from Tendrl POV? > > Or would an output like the following make more sense? Let us know. Thanks. > > [Volumes] > Volume1.name: tv1 > Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 > Volume1.type: Distribute > > Volume1.rebalance.files: 0 > Volume1.rebalance.data: 0Bytes > Volume1.options.nfs.disable: on > Volume1.options.performance.readdir-ahead: on > Volume1.options.transport.address-family: inet > Volume1.options.features.uss: on > > Volume2.name: tv2 > Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a > Volume2.type: Distribute > ......... > ......... > > > ~ Samikshan > -- ~ Atin (atinm) From shtripat at redhat.com Mon Nov 21 07:24:26 2016 From: shtripat at redhat.com (Shubhendu Tripathi) Date: Mon, 21 Nov 2016 12:54:26 +0530 Subject: [Tendrl-devel] Regarding some additional data points in the state output In-Reply-To: References: <5e1609e4-cfc3-4dc3-f421-ac51f56b9d64@redhat.com> Message-ID: <1988c0ba-6dfb-cdb3-a70e-4e51f02830e0@redhat.com> Looking at options, I feel option-2 would be more feasible and might not need code changes in tendrl. But still lets wait for the confirmation from Rohan. Regards, Shubhendu On 11/21/2016 12:43 PM, Atin Mukherjee wrote: > +tendrl-devel > > On Mon, Nov 21, 2016 at 12:41 PM, Samikshan Bairagya > wrote: > >> Hey Rohan, >> >> So the current get-state CLI misses volume specific options in its output. >> Somehow I missed it while coming up with the implementation. This patch by >> Atin is a fix for that: http://review.gluster.org/#/c/15889/1. The >> following example shows how this patch would add these new data points and >> how that would change the existing format: >> >> [Volumes] >> Volume1.name: tv1 >> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >> Volume1.type: Distribute >> >> Volume1.rebalance.files: 0 >> Volume1.rebalance.data: 0Bytes >> [Volume1.options] >> nfs.disable: on >> performance.readdir-ahead: on >> transport.address-family: inet >> features.uss: on >> >> Volume2.name: tv2 >> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >> Volume2.type: Distribute >> ......... >> ......... >> >> So essentially there would be a new section for every volume that would >> list the option names and corresponding values. Would adding this change >> still keep the get-state output parseable from Tendrl POV? >> >> Or would an output like the following make more sense? Let us know. Thanks. >> >> [Volumes] >> Volume1.name: tv1 >> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >> Volume1.type: Distribute >> >> Volume1.rebalance.files: 0 >> Volume1.rebalance.data: 0Bytes >> Volume1.options.nfs.disable: on >> Volume1.options.performance.readdir-ahead: on >> Volume1.options.transport.address-family: inet >> Volume1.options.features.uss: on >> >> Volume2.name: tv2 >> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >> Volume2.type: Distribute >> ......... >> ......... >> >> >> ~ Samikshan >> > > From adeza at redhat.com Mon Nov 21 12:56:01 2016 From: adeza at redhat.com (Alfredo Deza) Date: Mon, 21 Nov 2016 07:56:01 -0500 Subject: [Tendrl-devel] Provisioning Integration for Ceph and Gluster In-Reply-To: References: Message-ID: On Thu, Nov 17, 2016 at 8:10 AM, Mrugesh Karnik wrote: > On 14 November 2016 at 20:47, Alfredo Deza wrote: >> On Tue, Nov 8, 2016 at 7:39 AM, Mrugesh Karnik > wrote: >> > On 5 November 2016 at 01:51, Alfredo Deza wrote: >> >> We estimate that delivering all of the BZs isn't going to be possible >> >> by the end of November. Description on effort per BZ inline. >> > >> > Where would you put the estimate at, if not November end? >> >> To complete all BZs we are looking at roughly 1.5 months of Andrew and >> I working on them. >> >> At this point we don't think we can provide undivided attention to >> complete that effort. Since we are already two weeks into November >> we are already looking at early to mid December to complete this BZ: >> >> https://bugzilla.redhat.com/show_bug.cgi?id=1309415 > > The callbacks functionality isn't going to be required by Tendrl, polling > is sufficient. We went with the list of requirements for Tendrl and the callback was included. > Would it be possible for you to focus on the other issues, > instead, and provide us with a more concrete timeframe for by when work on > them could be completed? > It will not be possible to focus on the other items as they require significant effort. That is why we were able to commit only to callbacks earlier this month. The estimated level of effort is about 1.5 months for both me and Andrew to get the rest of BZs working. At this time we are unable to accommodate that amount of work. > Thanks, > Mrugesh > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From japplewh at redhat.com Mon Nov 21 13:26:05 2016 From: japplewh at redhat.com (Jeff Applewhite) Date: Mon, 21 Nov 2016 08:26:05 -0500 Subject: [Tendrl-devel] Provisioning Integration for Ceph and Gluster In-Reply-To: References: Message-ID: There is currently a fork of the calamari api supporting the Tendrl project and the need is to have the Calamari APIs support the same features in this fork (or at least what has been requested). This is a serious concern given our agreed mutual path forward. We need to re-prioritize or define a new path forward. On Mon, Nov 21, 2016 at 7:56 AM, Alfredo Deza wrote: > On Thu, Nov 17, 2016 at 8:10 AM, Mrugesh Karnik > wrote: > > On 14 November 2016 at 20:47, Alfredo Deza wrote: > >> On Tue, Nov 8, 2016 at 7:39 AM, Mrugesh Karnik > > > wrote: > >> > On 5 November 2016 at 01:51, Alfredo Deza wrote: > >> >> We estimate that delivering all of the BZs isn't going to be possible > >> >> by the end of November. Description on effort per BZ inline. > >> > > >> > Where would you put the estimate at, if not November end? > >> > >> To complete all BZs we are looking at roughly 1.5 months of Andrew and > >> I working on them. > >> > >> At this point we don't think we can provide undivided attention to > >> complete that effort. Since we are already two weeks into November > >> we are already looking at early to mid December to complete this BZ: > >> > >> https://bugzilla.redhat.com/show_bug.cgi?id=1309415 > > > > The callbacks functionality isn't going to be required by Tendrl, polling > > is sufficient. > > We went with the list of requirements for Tendrl and the callback was > included. > > > Would it be possible for you to focus on the other issues, > > instead, and provide us with a more concrete timeframe for by when work > on > > them could be completed? > > > > It will not be possible to focus on the other items as they require > significant effort. That is why we were able to commit > only to callbacks earlier this month. > > The estimated level of effort is about 1.5 months for both me and > Andrew to get the rest of BZs working. At this time we are unable to > accommodate that amount of work. > > > Thanks, > > Mrugesh > > _______________________________________________ > > Tendrl-devel mailing list > > Tendrl-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/tendrl-devel > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > -- Jeff Applewhite Principal Product Manager From adeza at redhat.com Mon Nov 21 15:12:20 2016 From: adeza at redhat.com (Alfredo Deza) Date: Mon, 21 Nov 2016 10:12:20 -0500 Subject: [Tendrl-devel] Provisioning Integration for Ceph and Gluster In-Reply-To: References: Message-ID: On Mon, Nov 21, 2016 at 8:26 AM, Jeff Applewhite wrote: > There is currently a fork of the calamari api supporting the Tendrl project > and the need is to have the Calamari APIs support the same features in this > fork (or at least what has been requested). I thought the discussion was about the ceph-installer, not Calamari. We weren't aware there was anything related with Calamari APIs other than deployment (which is currently supported) > This is a serious concern given > our agreed mutual path forward. We need to re-prioritize or define a new > path forward. For the ceph-installer we proposed the one BZ (#1309415) that we felt comfortable delivering by early December. Since it has been dropped, we can't commit to any of the other BZs for this year as they would require significant changes in how the ceph-installer operates. > > On Mon, Nov 21, 2016 at 7:56 AM, Alfredo Deza wrote: > >> On Thu, Nov 17, 2016 at 8:10 AM, Mrugesh Karnik >> wrote: >> > On 14 November 2016 at 20:47, Alfredo Deza wrote: >> >> On Tue, Nov 8, 2016 at 7:39 AM, Mrugesh Karnik > > >> > wrote: >> >> > On 5 November 2016 at 01:51, Alfredo Deza wrote: >> >> >> We estimate that delivering all of the BZs isn't going to be possible >> >> >> by the end of November. Description on effort per BZ inline. >> >> > >> >> > Where would you put the estimate at, if not November end? >> >> >> >> To complete all BZs we are looking at roughly 1.5 months of Andrew and >> >> I working on them. >> >> >> >> At this point we don't think we can provide undivided attention to >> >> complete that effort. Since we are already two weeks into November >> >> we are already looking at early to mid December to complete this BZ: >> >> >> >> https://bugzilla.redhat.com/show_bug.cgi?id=1309415 >> > >> > The callbacks functionality isn't going to be required by Tendrl, polling >> > is sufficient. >> >> We went with the list of requirements for Tendrl and the callback was >> included. >> >> > Would it be possible for you to focus on the other issues, >> > instead, and provide us with a more concrete timeframe for by when work >> on >> > them could be completed? >> > >> >> It will not be possible to focus on the other items as they require >> significant effort. That is why we were able to commit >> only to callbacks earlier this month. >> >> The estimated level of effort is about 1.5 months for both me and >> Andrew to get the rest of BZs working. At this time we are unable to >> accommodate that amount of work. >> >> > Thanks, >> > Mrugesh >> > _______________________________________________ >> > Tendrl-devel mailing list >> > Tendrl-devel at redhat.com >> > https://www.redhat.com/mailman/listinfo/tendrl-devel >> >> _______________________________________________ >> Tendrl-devel mailing list >> Tendrl-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/tendrl-devel >> > > > > -- > Jeff Applewhite > Principal Product Manager > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From mbukatov at redhat.com Mon Nov 21 16:08:10 2016 From: mbukatov at redhat.com (Martin Bukatovic) Date: Mon, 21 Nov 2016 17:08:10 +0100 Subject: [Tendrl-devel] compute resources necessary for tendrl In-Reply-To: References: Message-ID: <2d76c8ee-177b-b82b-b317-320e893f1c37@redhat.com> On 11/15/2016 04:40 PM, John Fulton wrote: > Do we have the resource requirements for Tendrl documented? > > I am looking into integrating Tendrl with TripleO [1] and will need to > get a small Tendrl server running in TripleO CI. Because TripleO CI is > resource constrained I'd like to get an idea how much RAM/CPU/Disk I can > get away for running Tendrl from the list so I can provide that detail > to the TripleO team. Ping. Any update here? -- Martin Bukatovic USM QE team From japplewh at redhat.com Mon Nov 21 16:58:18 2016 From: japplewh at redhat.com (Jeff Applewhite) Date: Mon, 21 Nov 2016 11:58:18 -0500 Subject: [Tendrl-devel] Provisioning Integration for Ceph and Gluster In-Reply-To: References: Message-ID: I just want to make sure we are in alignment on this (on Calamari). I heard until quite recently that this fork was still in use so it needs discussion. Can the Tendrl team please outline any remaining gaps? On Mon, Nov 21, 2016 at 10:12 AM, Alfredo Deza wrote: > On Mon, Nov 21, 2016 at 8:26 AM, Jeff Applewhite > wrote: > > There is currently a fork of the calamari api supporting the Tendrl > project > > and the need is to have the Calamari APIs support the same features in > this > > fork (or at least what has been requested). > > I thought the discussion was about the ceph-installer, not Calamari. > We weren't aware there was > anything related with Calamari APIs other than deployment (which is > currently supported) > > > This is a serious concern given > > our agreed mutual path forward. We need to re-prioritize or define a new > > path forward. > > For the ceph-installer we proposed the one BZ (#1309415) that we felt > comfortable delivering by > early December. Since it has been dropped, we can't commit to any of > the other BZs for this year as they would > require significant changes in how the ceph-installer operates. > > > > > > On Mon, Nov 21, 2016 at 7:56 AM, Alfredo Deza wrote: > > > >> On Thu, Nov 17, 2016 at 8:10 AM, Mrugesh Karnik < > mrugesh at brainfunked.org> > >> wrote: > >> > On 14 November 2016 at 20:47, Alfredo Deza wrote: > >> >> On Tue, Nov 8, 2016 at 7:39 AM, Mrugesh Karnik < > mrugesh at brainfunked.org > >> > > >> > wrote: > >> >> > On 5 November 2016 at 01:51, Alfredo Deza > wrote: > >> >> >> We estimate that delivering all of the BZs isn't going to be > possible > >> >> >> by the end of November. Description on effort per BZ inline. > >> >> > > >> >> > Where would you put the estimate at, if not November end? > >> >> > >> >> To complete all BZs we are looking at roughly 1.5 months of Andrew > and > >> >> I working on them. > >> >> > >> >> At this point we don't think we can provide undivided attention to > >> >> complete that effort. Since we are already two weeks into November > >> >> we are already looking at early to mid December to complete this BZ: > >> >> > >> >> https://bugzilla.redhat.com/show_bug.cgi?id=1309415 > >> > > >> > The callbacks functionality isn't going to be required by Tendrl, > polling > >> > is sufficient. > >> > >> We went with the list of requirements for Tendrl and the callback was > >> included. > >> > >> > Would it be possible for you to focus on the other issues, > >> > instead, and provide us with a more concrete timeframe for by when > work > >> on > >> > them could be completed? > >> > > >> > >> It will not be possible to focus on the other items as they require > >> significant effort. That is why we were able to commit > >> only to callbacks earlier this month. > >> > >> The estimated level of effort is about 1.5 months for both me and > >> Andrew to get the rest of BZs working. At this time we are unable to > >> accommodate that amount of work. > >> > >> > Thanks, > >> > Mrugesh > >> > _______________________________________________ > >> > Tendrl-devel mailing list > >> > Tendrl-devel at redhat.com > >> > https://www.redhat.com/mailman/listinfo/tendrl-devel > >> > >> _______________________________________________ > >> Tendrl-devel mailing list > >> Tendrl-devel at redhat.com > >> https://www.redhat.com/mailman/listinfo/tendrl-devel > >> > > > > > > > > -- > > Jeff Applewhite > > Principal Product Manager > > _______________________________________________ > > Tendrl-devel mailing list > > Tendrl-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/tendrl-devel > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > -- Jeff Applewhite Principal Product Manager From gmeno at redhat.com Mon Nov 21 20:38:53 2016 From: gmeno at redhat.com (Gregory Meno) Date: Mon, 21 Nov 2016 12:38:53 -0800 Subject: [Tendrl-devel] Tendrl API v1.0 upstream release In-Reply-To: References: Message-ID: This is awesome. Where can I get packages to try it out? cheers, Gregory On Thu, Nov 17, 2016 at 1:03 PM, Anup Nivargi wrote: > Hi, > > Tendrl API v1.0 has been released upstream [1] > > The release includes the initial framework setup along with a few storage > management API's like importing a Ceph cluster, importing a Gluster cluster, > creating/deleting a Ceph pool and creating/deleting a Gluster volume. > > It also provides basic features like to list nodes, clusters, volumes, peers and pools. > > [1] https://github.com/Tendrl/tendrl-api/releases/tag/v1.0 > > -- > Anup Nivargi > > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From japplewh at redhat.com Mon Nov 21 21:07:39 2016 From: japplewh at redhat.com (Jeff Applewhite) Date: Mon, 21 Nov 2016 16:07:39 -0500 Subject: [Tendrl-devel] Tendrl API v1.0 upstream release In-Reply-To: References: Message-ID: Yes this is great progress. *Also we need to get the APIs documented as we go so they can be used. We have integration work from other teams that will depend on this. Thanks! On Mon, Nov 21, 2016 at 3:38 PM, Gregory Meno wrote: > This is awesome. Where can I get packages to try it out? > > cheers, > Gregory > > On Thu, Nov 17, 2016 at 1:03 PM, Anup Nivargi wrote: > > Hi, > > > > Tendrl API v1.0 has been released upstream [1] > > > > The release includes the initial framework setup along with a few storage > > management API's like importing a Ceph cluster, importing a Gluster > cluster, > > creating/deleting a Ceph pool and creating/deleting a Gluster volume. > > > > It also provides basic features like to list nodes, clusters, volumes, > peers and pools. > > > > [1] https://github.com/Tendrl/tendrl-api/releases/tag/v1.0 > > > > -- > > Anup Nivargi > > > > > > _______________________________________________ > > Tendrl-devel mailing list > > Tendrl-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/tendrl-devel > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > -- Jeff Applewhite Principal Product Manager From nthomas at redhat.com Tue Nov 22 03:15:13 2016 From: nthomas at redhat.com (Nishanth Thomas) Date: Tue, 22 Nov 2016 08:45:13 +0530 Subject: [Tendrl-devel] compute resources necessary for tendrl In-Reply-To: <2d76c8ee-177b-b82b-b317-320e893f1c37@redhat.com> References: <2d76c8ee-177b-b82b-b317-320e893f1c37@redhat.com> Message-ID: Here is what I am thinking about(in-line with what is proposed for skyring ) * quad core CPU(Or multiple dual core CPUs) * 16 GB of RAM * One NIC with at-least 1 Gbps bandwidth thoughts? On Mon, Nov 21, 2016 at 9:38 PM, Martin Bukatovic wrote: > On 11/15/2016 04:40 PM, John Fulton wrote: > > Do we have the resource requirements for Tendrl documented? > > > > I am looking into integrating Tendrl with TripleO [1] and will need to > > get a small Tendrl server running in TripleO CI. Because TripleO CI is > > resource constrained I'd like to get an idea how much RAM/CPU/Disk I can > > get away for running Tendrl from the list so I can provide that detail > > to the TripleO team. > > Ping. Any update here? > > -- > Martin Bukatovic > USM QE team > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From amukherj at redhat.com Tue Nov 22 03:48:04 2016 From: amukherj at redhat.com (Atin Mukherjee) Date: Tue, 22 Nov 2016 09:18:04 +0530 Subject: [Tendrl-devel] Regarding some additional data points in the state output In-Reply-To: <1988c0ba-6dfb-cdb3-a70e-4e51f02830e0@redhat.com> References: <5e1609e4-cfc3-4dc3-f421-ac51f56b9d64@redhat.com> <1988c0ba-6dfb-cdb3-a70e-4e51f02830e0@redhat.com> Message-ID: We are awaiting final confirmation from Rohan. Samikshan has kept the changes ready and will push it to gerrit once we hear from Rohan. On Mon, Nov 21, 2016 at 12:54 PM, Shubhendu Tripathi wrote: > Looking at options, I feel option-2 would be more feasible and might not > need code changes in tendrl. > But still lets wait for the confirmation from Rohan. > > Regards, > Shubhendu > > > > On 11/21/2016 12:43 PM, Atin Mukherjee wrote: > >> +tendrl-devel >> >> On Mon, Nov 21, 2016 at 12:41 PM, Samikshan Bairagya > > >> wrote: >> >> Hey Rohan, >>> >>> So the current get-state CLI misses volume specific options in its >>> output. >>> Somehow I missed it while coming up with the implementation. This patch >>> by >>> Atin is a fix for that: http://review.gluster.org/#/c/15889/1. The >>> following example shows how this patch would add these new data points >>> and >>> how that would change the existing format: >>> >>> [Volumes] >>> Volume1.name: tv1 >>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>> Volume1.type: Distribute >>> >>> Volume1.rebalance.files: 0 >>> Volume1.rebalance.data: 0Bytes >>> [Volume1.options] >>> nfs.disable: on >>> performance.readdir-ahead: on >>> transport.address-family: inet >>> features.uss: on >>> >>> Volume2.name: tv2 >>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>> Volume2.type: Distribute >>> ......... >>> ......... >>> >>> So essentially there would be a new section for every volume that would >>> list the option names and corresponding values. Would adding this change >>> still keep the get-state output parseable from Tendrl POV? >>> >>> Or would an output like the following make more sense? Let us know. >>> Thanks. >>> >>> [Volumes] >>> Volume1.name: tv1 >>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>> Volume1.type: Distribute >>> >>> Volume1.rebalance.files: 0 >>> Volume1.rebalance.data: 0Bytes >>> Volume1.options.nfs.disable: on >>> Volume1.options.performance.readdir-ahead: on >>> Volume1.options.transport.address-family: inet >>> Volume1.options.features.uss: on >>> >>> Volume2.name: tv2 >>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>> Volume2.type: Distribute >>> ......... >>> ......... >>> >>> >>> ~ Samikshan >>> >>> >> >> > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > -- ~ Atin (atinm) From avishwan at redhat.com Tue Nov 22 06:24:08 2016 From: avishwan at redhat.com (Aravinda) Date: Tue, 22 Nov 2016 11:54:08 +0530 Subject: [Tendrl-devel] Gluster Events APIs integration with Tendrl Message-ID: <7fdd4d17-d35e-0968-cce7-45a8bf2945b4@redhat.com> Hi, Gluster now has support for real time notifications with Release 3.9. Applications/Clients can subscribe to Events by registering as Webhook. More details about Events APIs are available in the following link http://gluster.readthedocs.io/en/latest/Administrator%20Guide/Events%20APIs Based on the initial discussion with Tendrl team, Gluster Events are listened by gluster-bridge and refreshed state based on the Event. Since this is new feature from Gluster, we may need more enhancements to Events APIs to integrate with Tendrl. Please let us know if any progress made with Gluster Events APIs integration. -- regards Aravinda http://aravindavk.in From rkanade at redhat.com Tue Nov 22 08:14:33 2016 From: rkanade at redhat.com (Rohan Kanade) Date: Tue, 22 Nov 2016 13:44:33 +0530 Subject: [Tendrl-devel] Regarding some additional data points in the state output In-Reply-To: References: <5e1609e4-cfc3-4dc3-f421-ac51f56b9d64@redhat.com> <1988c0ba-6dfb-cdb3-a70e-4e51f02830e0@redhat.com> Message-ID: Id prefer the first option [Volumes] Volume1.name: tv1 Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 Volume1.type: Distribute Volume1.rebalance.files: 0 Volume1.rebalance.data: 0Bytes [Volume1.options] nfs.disable: on performance.readdir-ahead: on transport.address-family: inet features.uss: on Volume2.name: tv2 Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a Volume2.type: Distribute ......... ......... This would require minor changes to tendrl/gluster-integration definition files and code. I will draw up a spec on Tendrl/specifications to document the changes required. Please go ahead with your patch Thanks On Tue, Nov 22, 2016 at 9:18 AM, Atin Mukherjee wrote: > We are awaiting final confirmation from Rohan. Samikshan has kept the > changes ready and will push it to gerrit once we hear from Rohan. > > On Mon, Nov 21, 2016 at 12:54 PM, Shubhendu Tripathi > wrote: > > > Looking at options, I feel option-2 would be more feasible and might not > > need code changes in tendrl. > > But still lets wait for the confirmation from Rohan. > > > > Regards, > > Shubhendu > > > > > > > > On 11/21/2016 12:43 PM, Atin Mukherjee wrote: > > > >> +tendrl-devel > >> > >> On Mon, Nov 21, 2016 at 12:41 PM, Samikshan Bairagya < > sbairagy at redhat.com > >> > > >> wrote: > >> > >> Hey Rohan, > >>> > >>> So the current get-state CLI misses volume specific options in its > >>> output. > >>> Somehow I missed it while coming up with the implementation. This patch > >>> by > >>> Atin is a fix for that: http://review.gluster.org/#/c/15889/1. The > >>> following example shows how this patch would add these new data points > >>> and > >>> how that would change the existing format: > >>> > >>> [Volumes] > >>> Volume1.name: tv1 > >>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 > >>> Volume1.type: Distribute > >>> > >>> Volume1.rebalance.files: 0 > >>> Volume1.rebalance.data: 0Bytes > >>> [Volume1.options] > >>> nfs.disable: on > >>> performance.readdir-ahead: on > >>> transport.address-family: inet > >>> features.uss: on > >>> > >>> Volume2.name: tv2 > >>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a > >>> Volume2.type: Distribute > >>> ......... > >>> ......... > >>> > >>> So essentially there would be a new section for every volume that would > >>> list the option names and corresponding values. Would adding this > change > >>> still keep the get-state output parseable from Tendrl POV? > >>> > >>> Or would an output like the following make more sense? Let us know. > >>> Thanks. > >>> > >>> [Volumes] > >>> Volume1.name: tv1 > >>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 > >>> Volume1.type: Distribute > >>> > >>> Volume1.rebalance.files: 0 > >>> Volume1.rebalance.data: 0Bytes > >>> Volume1.options.nfs.disable: on > >>> Volume1.options.performance.readdir-ahead: on > >>> Volume1.options.transport.address-family: inet > >>> Volume1.options.features.uss: on > >>> > >>> Volume2.name: tv2 > >>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a > >>> Volume2.type: Distribute > >>> ......... > >>> ......... > >>> > >>> > >>> ~ Samikshan > >>> > >>> > >> > >> > > _______________________________________________ > > Tendrl-devel mailing list > > Tendrl-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/tendrl-devel > > > > > > -- > > ~ Atin (atinm) > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From rkanade at redhat.com Tue Nov 22 08:36:34 2016 From: rkanade at redhat.com (Rohan Kanade) Date: Tue, 22 Nov 2016 14:06:34 +0530 Subject: [Tendrl-devel] Regarding some additional data points in the state output In-Reply-To: References: <5e1609e4-cfc3-4dc3-f421-ac51f56b9d64@redhat.com> <1988c0ba-6dfb-cdb3-a70e-4e51f02830e0@redhat.com> Message-ID: Also, please provide a full state dump example with this patch included, easier for Tendrl devs to get started without deploying this patch On Tue, Nov 22, 2016 at 1:44 PM, Rohan Kanade wrote: > Id prefer the first option > > > [Volumes] > Volume1.name: tv1 > Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 > Volume1.type: Distribute > > Volume1.rebalance.files: 0 > Volume1.rebalance.data: 0Bytes > [Volume1.options] > nfs.disable: on > performance.readdir-ahead: on > transport.address-family: inet > features.uss: on > > Volume2.name: tv2 > Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a > Volume2.type: Distribute > ......... > ......... > > > This would require minor changes to tendrl/gluster-integration definition > files and code. I will draw up a spec on Tendrl/specifications to document > the changes required. Please go ahead with your patch > Thanks > > On Tue, Nov 22, 2016 at 9:18 AM, Atin Mukherjee > wrote: > >> We are awaiting final confirmation from Rohan. Samikshan has kept the >> changes ready and will push it to gerrit once we hear from Rohan. >> >> On Mon, Nov 21, 2016 at 12:54 PM, Shubhendu Tripathi > > >> wrote: >> >> > Looking at options, I feel option-2 would be more feasible and might not >> > need code changes in tendrl. >> > But still lets wait for the confirmation from Rohan. >> > >> > Regards, >> > Shubhendu >> > >> > >> > >> > On 11/21/2016 12:43 PM, Atin Mukherjee wrote: >> > >> >> +tendrl-devel >> >> >> >> On Mon, Nov 21, 2016 at 12:41 PM, Samikshan Bairagya < >> sbairagy at redhat.com >> >> > >> >> wrote: >> >> >> >> Hey Rohan, >> >>> >> >>> So the current get-state CLI misses volume specific options in its >> >>> output. >> >>> Somehow I missed it while coming up with the implementation. This >> patch >> >>> by >> >>> Atin is a fix for that: http://review.gluster.org/#/c/15889/1. The >> >>> following example shows how this patch would add these new data points >> >>> and >> >>> how that would change the existing format: >> >>> >> >>> [Volumes] >> >>> Volume1.name: tv1 >> >>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >> >>> Volume1.type: Distribute >> >>> >> >>> Volume1.rebalance.files: 0 >> >>> Volume1.rebalance.data: 0Bytes >> >>> [Volume1.options] >> >>> nfs.disable: on >> >>> performance.readdir-ahead: on >> >>> transport.address-family: inet >> >>> features.uss: on >> >>> >> >>> Volume2.name: tv2 >> >>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >> >>> Volume2.type: Distribute >> >>> ......... >> >>> ......... >> >>> >> >>> So essentially there would be a new section for every volume that >> would >> >>> list the option names and corresponding values. Would adding this >> change >> >>> still keep the get-state output parseable from Tendrl POV? >> >>> >> >>> Or would an output like the following make more sense? Let us know. >> >>> Thanks. >> >>> >> >>> [Volumes] >> >>> Volume1.name: tv1 >> >>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >> >>> Volume1.type: Distribute >> >>> >> >>> Volume1.rebalance.files: 0 >> >>> Volume1.rebalance.data: 0Bytes >> >>> Volume1.options.nfs.disable: on >> >>> Volume1.options.performance.readdir-ahead: on >> >>> Volume1.options.transport.address-family: inet >> >>> Volume1.options.features.uss: on >> >>> >> >>> Volume2.name: tv2 >> >>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >> >>> Volume2.type: Distribute >> >>> ......... >> >>> ......... >> >>> >> >>> >> >>> ~ Samikshan >> >>> >> >>> >> >> >> >> >> > _______________________________________________ >> > Tendrl-devel mailing list >> > Tendrl-devel at redhat.com >> > https://www.redhat.com/mailman/listinfo/tendrl-devel >> > >> >> >> >> -- >> >> ~ Atin (atinm) >> _______________________________________________ >> Tendrl-devel mailing list >> Tendrl-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/tendrl-devel >> > > From mbukatov at redhat.com Tue Nov 22 08:39:58 2016 From: mbukatov at redhat.com (Martin Bukatovic) Date: Tue, 22 Nov 2016 09:39:58 +0100 Subject: [Tendrl-devel] compute resources necessary for tendrl In-Reply-To: References: <2d76c8ee-177b-b82b-b317-320e893f1c37@redhat.com> Message-ID: On 11/22/2016 04:15 AM, Nishanth Thomas wrote: > Here is what I am thinking about(in-line with what is proposed for skyring ) > > * quad core CPU(Or multiple dual core CPUs) > * 16 GB of RAM > * One NIC with at-least 1 Gbps bandwidth Do I read this right as a requirement list for tendrl machine (where the tendrl management console is isntalled). At first sight, these numbers looks a bit overblown. Could you elaborate? Are those numbers minimal or optimal? Also, do we need to specify requirements for other cluster roles as well? -- Martin Bukatovic USM QE team From ltrilety at redhat.com Tue Nov 22 08:40:51 2016 From: ltrilety at redhat.com (Lubos Trilety) Date: Tue, 22 Nov 2016 09:40:51 +0100 Subject: [Tendrl-devel] compute resources necessary for tendrl In-Reply-To: References: <2d76c8ee-177b-b82b-b317-320e893f1c37@redhat.com> Message-ID: Do we really have such big 'minimum hardware requirements'? Are you aware that we will probably never test that with such configuration not speaking about a bigger one. And even so it will work. Such requirements could be for optimal work, but I don?t think that such configuration is the minimum which is necessary for tendrl to be working. Well, that?s just my humble opinion, I could be wrong. > On 22 Nov 2016, at 04:15, Nishanth Thomas wrote: > > Here is what I am thinking about(in-line with what is proposed for skyring ) > > * quad core CPU(Or multiple dual core CPUs) > * 16 GB of RAM > * One NIC with at-least 1 Gbps bandwidth > > thoughts? > > On Mon, Nov 21, 2016 at 9:38 PM, Martin Bukatovic > wrote: > >> On 11/15/2016 04:40 PM, John Fulton wrote: >>> Do we have the resource requirements for Tendrl documented? >>> >>> I am looking into integrating Tendrl with TripleO [1] and will need to >>> get a small Tendrl server running in TripleO CI. Because TripleO CI is >>> resource constrained I'd like to get an idea how much RAM/CPU/Disk I can >>> get away for running Tendrl from the list so I can provide that detail >>> to the TripleO team. >> >> Ping. Any update here? >> >> -- >> Martin Bukatovic >> USM QE team >> >> _______________________________________________ >> Tendrl-devel mailing list >> Tendrl-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/tendrl-devel >> > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From nthomas at redhat.com Tue Nov 22 09:22:28 2016 From: nthomas at redhat.com (Nishanth Thomas) Date: Tue, 22 Nov 2016 14:52:28 +0530 Subject: [Tendrl-devel] compute resources necessary for tendrl In-Reply-To: References: <2d76c8ee-177b-b82b-b317-320e893f1c37@redhat.com> Message-ID: Martin/Lubos, Yes, these are the requirements for server machine where the etcd, tendrl-api are hosted. Also the monitoring application(optional component- brings in graphite and whisper as a dependency) also might be running here. What I mentioned here is the optimal configuration(in production) and I think John is asking for the same. For requirements for the cluster nodes, one should look at the spec published by the storage providers(ceph, gluster etc). Tendrl services are supposed to put on a little overhead on those nodes. Thanks, Nishanth On Tue, Nov 22, 2016 at 2:10 PM, Lubos Trilety wrote: > Do we really have such big 'minimum hardware requirements'? > Are you aware that we will probably never test that with such > configuration not speaking about a bigger one. And even so it will work. > Such requirements could be for optimal work, but I don?t think that such > configuration is the minimum which is necessary for tendrl to be working. > Well, that?s just my humble opinion, I could be wrong. > > > On 22 Nov 2016, at 04:15, Nishanth Thomas wrote: > > > > Here is what I am thinking about(in-line with what is proposed for > skyring ) > > > > * quad core CPU(Or multiple dual core CPUs) > > * 16 GB of RAM > > * One NIC with at-least 1 Gbps bandwidth > > > > thoughts? > > > > On Mon, Nov 21, 2016 at 9:38 PM, Martin Bukatovic > > wrote: > > > >> On 11/15/2016 04:40 PM, John Fulton wrote: > >>> Do we have the resource requirements for Tendrl documented? > >>> > >>> I am looking into integrating Tendrl with TripleO [1] and will need to > >>> get a small Tendrl server running in TripleO CI. Because TripleO CI is > >>> resource constrained I'd like to get an idea how much RAM/CPU/Disk I > can > >>> get away for running Tendrl from the list so I can provide that detail > >>> to the TripleO team. > >> > >> Ping. Any update here? > >> > >> -- > >> Martin Bukatovic > >> USM QE team > >> > >> _______________________________________________ > >> Tendrl-devel mailing list > >> Tendrl-devel at redhat.com > >> https://www.redhat.com/mailman/listinfo/tendrl-devel > >> > > _______________________________________________ > > Tendrl-devel mailing list > > Tendrl-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/tendrl-devel > > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From jspray at redhat.com Tue Nov 22 10:35:36 2016 From: jspray at redhat.com (John Spray) Date: Tue, 22 Nov 2016 10:35:36 +0000 Subject: [Tendrl-devel] compute resources necessary for tendrl In-Reply-To: References: <2d76c8ee-177b-b82b-b317-320e893f1c37@redhat.com> Message-ID: On Tue, Nov 22, 2016 at 3:15 AM, Nishanth Thomas wrote: > Here is what I am thinking about(in-line with what is proposed for skyring ) > > * quad core CPU(Or multiple dual core CPUs) > * 16 GB of RAM > * One NIC with at-least 1 Gbps bandwidth > > thoughts? Is the expectation to have 3x this configuration for HA? I posted a while back ("Deployment/dependency questions") to ask whether the intention was to run the tendrl servers on the same servers as Ceph mons or on separate servers, but never got a response. John > On Mon, Nov 21, 2016 at 9:38 PM, Martin Bukatovic > wrote: > >> On 11/15/2016 04:40 PM, John Fulton wrote: >> > Do we have the resource requirements for Tendrl documented? >> > >> > I am looking into integrating Tendrl with TripleO [1] and will need to >> > get a small Tendrl server running in TripleO CI. Because TripleO CI is >> > resource constrained I'd like to get an idea how much RAM/CPU/Disk I can >> > get away for running Tendrl from the list so I can provide that detail >> > to the TripleO team. >> >> Ping. Any update here? >> >> -- >> Martin Bukatovic >> USM QE team >> >> _______________________________________________ >> Tendrl-devel mailing list >> Tendrl-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/tendrl-devel >> > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From mbukatov at redhat.com Tue Nov 22 12:18:10 2016 From: mbukatov at redhat.com (Martin Bukatovic) Date: Tue, 22 Nov 2016 13:18:10 +0100 Subject: [Tendrl-devel] [Release] Tendrl v1.0 In-Reply-To: References: Message-ID: On 11/17/2016 07:38 AM, Rohan Kanade wrote: > We are pleased to announce the release of Tendrl v1.0. > > > The first release brings capabilities for importing Ceph and Gluster > clusters and primary CRUD operations on cluster objects like Volumes, > Peers, Pools etc. I see a little discrepancies among our tracking tools, documentation and actual status. To be able to go on, we need to review those gaps in the management and try to fix them right now to prevent similar problems in the future. First, I see the following definition of the release in our Developer Workflow document[1]: > Tendrl Core aims to have bi-monthly releases complete with testing, > documentation and consumable via packages. While this Tendrl v1.0 has been declared without any of those. Moreover, the dev work flow document[1] also states that: > Jira would be used for feature tracking in the current and > future releases. But when I look into JIRA to see what is part of the v1.0 release, I see it as not yet released, with most of it's task not done[3]. That overview also doesn't match this wiki page for release v1.0[2] where there are other list of jira tasks, with only few in accepted state. In other words, most of the tasks are not done here as well. [1] https://github.com/Tendrl/documentation/blob/master/development-workflow.adoc [2] https://tendrl.atlassian.net/wiki/display/UD/Tendrl+Release+One [3] https://tendrl.atlassian.net/projects/TEN/versions/10100/tab/release-report-all-issues -- Martin Bukatovic USM QE team From mkudlej at redhat.com Tue Nov 22 12:33:18 2016 From: mkudlej at redhat.com (Martin Kudlej) Date: Tue, 22 Nov 2016 13:33:18 +0100 Subject: [Tendrl-devel] [Release] Tendrl v1.0 In-Reply-To: References: Message-ID: Hi Rohan, comments inline. On 11/17/2016 07:38 AM, Rohan Kanade wrote: > We are pleased to announce the release of Tendrl v1.0. congratulations! > The first release brings capabilities for importing Ceph and Gluster > clusters and primary CRUD operations on cluster objects like Volumes, > Peers, Pools etc. I would like to ask where is complete list of features included in this release? Is there any other place(wiki page, solution on top of Github, something else) where Tendrl community do release planning? We(QE) would like to be part of this planning. It seems to me that Jira[1] is out of date. Also I've seen [3], [4], [5] and they aren't synchronized. Also I would like to ask if DoD [2] is still valid? Was there any changes? Should Tendrl 1 features follow all DoD? I'm open to discussion about this so we can enhance DoD to be better aligned to Tendrl team and project needs. It seems to me that communication in community is not perfect and it should be enhanced. Especially communication about project planning. I think that we need to have Scrum master role in team if we would like to follow agile-like workflow. It seems to me now that our workflow is closer to Waterfall than Agile-like workflow. We work in team which is spread around the world so I think we should have in team somebody who will take care of team members. Majority of team is in India so I think it is logical to have Scrum master there. Please write what do you think about this. I'm open for discussion and I would like to enhance communication so we work on Tendrl as one team. [1] https://tendrl.atlassian.net/secure/RapidBoard.jspa?rapidView=4 [2] https://github.com/Tendrl/documentation/issues/8 [3] https://tendrl.atlassian.net/wiki/display/UD/Tendrl+Release+One [4] https://tendrl.atlassian.net/projects/TEN?selectedItem=com.atlassian.jira.jira-projects-plugin%3Arelease-page&status=unreleased [5] https://www.redhat.com/archives/tendrl-devel/2016-November/msg00053.html -- Best Regards, Martin Kudlej. RHSC/USM Senior Quality Assurance Engineer Red Hat Czech s.r.o. Phone: +420 532 294 155 E-mail:mkudlej at redhat.com IRC: mkudlej at #brno, #gluster, #storage-qa, #rhs, #rh-ceph, #usm-meeting @ redhat #tendrl-devel @ freenode From mrugesh at brainfunked.org Tue Nov 22 13:03:31 2016 From: mrugesh at brainfunked.org (Mrugesh Karnik) Date: Tue, 22 Nov 2016 18:33:31 +0530 Subject: [Tendrl-devel] compute resources necessary for tendrl In-Reply-To: References: <2d76c8ee-177b-b82b-b317-320e893f1c37@redhat.com> Message-ID: On 22 November 2016 at 08:45, Nishanth Thomas wrote: > > Here is what I am thinking about(in-line with what is proposed for skyring ) > > * quad core CPU(Or multiple dual core CPUs) > * 16 GB of RAM > * One NIC with at-least 1 Gbps bandwidth > > thoughts? Now that we have a working build, I think we need to monitor and draw up the actual numbers for resource utilisation by our various components. Given that the components are more spread out, the original skyring based numbers may not be fully valid. -- Mrugesh From sankarshan.mukhopadhyay at gmail.com Tue Nov 22 14:34:44 2016 From: sankarshan.mukhopadhyay at gmail.com (Sankarshan Mukhopadhyay) Date: Tue, 22 Nov 2016 20:04:44 +0530 Subject: [Tendrl-devel] [Release] Tendrl v1.0 In-Reply-To: References: Message-ID: On Tue, Nov 22, 2016 at 6:03 PM, Martin Kudlej wrote: > Hi Rohan, > > comments inline. > > On 11/17/2016 07:38 AM, Rohan Kanade wrote: >> >> We are pleased to announce the release of Tendrl v1.0. > > congratulations! > > >> The first release brings capabilities for importing Ceph and Gluster >> clusters and primary CRUD operations on cluster objects like Volumes, >> Peers, Pools etc. > > > I would like to ask where is complete list of features included in this > release? > Is there any other place(wiki page, solution on top of Github, something > else) where Tendrl > community do release planning? > We(QE) would like to be part of this planning. I have a different question and a fresh approach to this. The notion of using Github issues to track PRs and Jira to track the stories was based on ensuring a cross-team participation and collaboration. If there is a lack of involvement in the release planning discussions, this needs to be revisited. A cursory look at the meetings and conversations seem to indicate that individuals with domain knowledge in testing (I use this long form as a synonym for QE, which is really a product specific function) have engaged in conversations. What has happened however is that the complement to unit testing ie. functional testing paths have not been added to the tasks/stories. In a manner of speaking, we do not have acceptance tests which can help us assess the state of each of the capabilities that are part of the build which has been tagged. Developers have spent a few cycles doing testing - but that is not expected to be a substitute for an user/admin persona driven testing. > It seems to me that Jira[1] is out of date. > Also I've seen [3], [4], [5] and they aren't synchronized. > Allow me to turn this around - contributors to this project are expected to collaborate. Developers will provide code and those with skills in testing would be bringing their expertise to ensure we have good releases to ship. If the baseline data sources which drive this process are out-of-whack, why aren't they being updated? Do you not have permissions/rights to do so? Or, is this a form of expectation mis-match that is causing an impediment? > Also I would like to ask if DoD [2] is still valid? Was there any changes? > Should Tendrl 1 features follow all DoD? > I'm open to discussion about this so we can enhance DoD to be better aligned > to Tendrl team and project needs. > The 'Definition of Done' is a guideline. At this stage of the project it encapsulates more of the aspiration ('where we should be') rather than reality ('where we are'). A reason it is such because it implicitly assumes there is a close-quarter collaboration in development-testing-documentation. From outside-in, as I look at the project, I do not see enough evidence of that. So, the DoD can only be a gate when we have tests running across builds. This isn't the first time I've stated this sentiment. But I certainly find it challenging to use the DoD gate when we aren't running functional tests on a build. We need to scope and assess the time and effort investment required to make tests a reality. Unless we have tests running on builds as automated jobs, it is harder to demonstrate a level of confidence to potential consumers of the project artefacts. This is as good a time as any to have a discussion around that topic and put a plan in place. > It seems to me that communication in community is not perfect and it should > be enhanced. Especially communication about project planning. I think that > we need to have Scrum master role in team if we would like to follow > agile-like workflow. It seems to me now that our workflow is closer to > Waterfall than Agile-like workflow. I would disagree. I would like to think that between this list; the establishment of office hours; the formulation of sprints and execution of them - we have a good enough rhythm in place that relies on communication. Distributed teams place unique challenges on projects. However, distributed (not 'remote') teams are a reality - in fact, every project has distributed teams (as a general rule of thumb). Quality and consistency of communication can always be improved - this is an iterative process. But it requires everyone to chip in. When we started the project and made the first few (hesitant?) steps towards using the list, there was a lot of exhorting to do better and default to open. If we aren't doing well on that - we have to improve. I take your feedback in such spirit. And I seek your help in proposing how we can do that. > We work in team which is spread around the world so I think we should have > in team somebody who will take care of team members. Majority of team is in > India so I think it is logical to have Scrum master there. > Please write what do you think about this. I'm open for discussion and I > would like to enhance communication so we work on Tendrl as one team. > > > [1] https://tendrl.atlassian.net/secure/RapidBoard.jspa?rapidView=4 > [2] https://github.com/Tendrl/documentation/issues/8 > [3] https://tendrl.atlassian.net/wiki/display/UD/Tendrl+Release+One > [4] > https://tendrl.atlassian.net/projects/TEN?selectedItem=com.atlassian.jira.jira-projects-plugin%3Arelease-page&status=unreleased > [5] https://www.redhat.com/archives/tendrl-devel/2016-November/msg00053.html > -- > Best Regards, > Martin Kudlej. > RHSC/USM Senior Quality Assurance Engineer > Red Hat Czech s.r.o. > > Phone: +420 532 294 155 > E-mail:mkudlej at redhat.com > IRC: mkudlej at #brno, #gluster, #storage-qa, #rhs, #rh-ceph, #usm-meeting > @ redhat > #tendrl-devel @ freenode > -- sankarshan mukhopadhyay From rkanade at redhat.com Tue Nov 22 14:46:00 2016 From: rkanade at redhat.com (Rohan Kanade) Date: Tue, 22 Nov 2016 20:16:00 +0530 Subject: [Tendrl-devel] Regarding some additional data points in the state output In-Reply-To: References: <5e1609e4-cfc3-4dc3-f421-ac51f56b9d64@redhat.com> <1988c0ba-6dfb-cdb3-a70e-4e51f02830e0@redhat.com> Message-ID: Sample: START>>> [Global] MYUUID: 6bbf8ac2-22a0-4f08-b986-fe75aea9f654 op-version: 40000 [Global options] [Peers] [Volumes] Volume1.name: test-vol Volume1.id: 7942d008-e300-4fd9-8af0-5a118afd8d3d Volume1.type: Distribute Volume1.transport_type: tcp Volume1.status: Started Volume1.brickcount: 1 Volume1.Brick1.path: 172.17.0.2:/tmp/b1 Volume1.Brick1.hostname: 172.17.0.2 Volume1.Brick1.port: 49153 Volume1.Brick1.rdma_port: 0 Volume1.Brick1.status: Started Volume1.Brick1.signedin: True Volume1.snap_count: 0 Volume1.stripe_count: 1 Volume1.replica_count: 1 Volume1.subvol_count: 1 Volume1.arbiter_count: 0 Volume1.disperse_count: 0 Volume1.redundancy_count: 0 Volume1.quorum_status: not_applicable Volume1.snapd_svc.online_status: Offline Volume1.snapd_svc.inited: True Volume1.rebalance.id: 00000000-0000-0000-0000-000000000000 Volume1.rebalance.status: not_started Volume1.rebalance.failures: 0 Volume1.rebalance.skipped: 0 Volume1.rebalance.lookedup: 0 Volume1.rebalance.files: 0 Volume1.rebalance.data: 0Bytes [Volume1.options] features.barrier: on transport.address-family: inet performance.readdir-ahead: on nfs.disable: on Volume2.name: test-vol1 Volume2.id: 35854708-bb72-45a5-bdbd-77c51e5ebfb9 Volume2.type: Distribute Volume2.transport_type: tcp Volume2.status: Started Volume2.brickcount: 1 Volume2.Brick1.path: 172.17.0.2:/tmp/b2 Volume2.Brick1.hostname: 172.17.0.2 Volume2.Brick1.port: 49152 Volume2.Brick1.rdma_port: 0 Volume2.Brick1.status: Started Volume2.Brick1.signedin: True Volume2.snap_count: 0 Volume2.stripe_count: 1 Volume2.replica_count: 1 Volume2.subvol_count: 1 Volume2.arbiter_count: 0 Volume2.disperse_count: 0 Volume2.redundancy_count: 0 Volume2.quorum_status: not_applicable Volume2.snapd_svc.online_status: Offline Volume2.snapd_svc.inited: True Volume2.rebalance.id: 00000000-0000-0000-0000-000000000000 Volume2.rebalance.status: not_started Volume2.rebalance.failures: 0 Volume2.rebalance.skipped: 0 Volume2.rebalance.lookedup: 0 Volume2.rebalance.files: 0 Volume2.rebalance.data: 0Bytes [Volume2.options] transport.address-family: inet performance.readdir-ahead: on nfs.disable: on [Services] svc1.name: glustershd svc1.online_status: Offline svc2.name: nfs svc2.online_status: Offline svc3.name: bitd svc3.online_status: Offline svc4.name: scrub svc4.online_status: Offline svc5.name: quotad svc5.online_status: Offline [Misc] Base port: 49152 Last allocated port: 49153 < wrote: > > Also, please provide a full state dump example with this patch included, easier for Tendrl devs to get started without deploying this patch > > On Tue, Nov 22, 2016 at 1:44 PM, Rohan Kanade wrote: >> >> Id prefer the first option >> >> >> [Volumes] >> Volume1.name: tv1 >> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >> Volume1.type: Distribute >> >> Volume1.rebalance.files: 0 >> Volume1.rebalance.data: 0Bytes >> [Volume1.options] >> nfs.disable: on >> performance.readdir-ahead: on >> transport.address-family: inet >> features.uss: on >> >> Volume2.name: tv2 >> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >> Volume2.type: Distribute >> ......... >> ......... >> >> >> This would require minor changes to tendrl/gluster-integration definition files and code. I will draw up a spec on Tendrl/specifications to document the changes required. Please go ahead with your patch >> Thanks >> >> On Tue, Nov 22, 2016 at 9:18 AM, Atin Mukherjee wrote: >>> >>> We are awaiting final confirmation from Rohan. Samikshan has kept the >>> changes ready and will push it to gerrit once we hear from Rohan. >>> >>> On Mon, Nov 21, 2016 at 12:54 PM, Shubhendu Tripathi < shtripat at redhat.com> >>> wrote: >>> >>> > Looking at options, I feel option-2 would be more feasible and might not >>> > need code changes in tendrl. >>> > But still lets wait for the confirmation from Rohan. >>> > >>> > Regards, >>> > Shubhendu >>> > >>> > >>> > >>> > On 11/21/2016 12:43 PM, Atin Mukherjee wrote: >>> > >>> >> +tendrl-devel >>> >> >>> >> On Mon, Nov 21, 2016 at 12:41 PM, Samikshan Bairagya < sbairagy at redhat.com >>> >> > >>> >> wrote: >>> >> >>> >> Hey Rohan, >>> >>> >>> >>> So the current get-state CLI misses volume specific options in its >>> >>> output. >>> >>> Somehow I missed it while coming up with the implementation. This patch >>> >>> by >>> >>> Atin is a fix for that: http://review.gluster.org/#/c/15889/1. The >>> >>> following example shows how this patch would add these new data points >>> >>> and >>> >>> how that would change the existing format: >>> >>> >>> >>> [Volumes] >>> >>> Volume1.name: tv1 >>> >>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>> >>> Volume1.type: Distribute >>> >>> >>> >>> Volume1.rebalance.files: 0 >>> >>> Volume1.rebalance.data: 0Bytes >>> >>> [Volume1.options] >>> >>> nfs.disable: on >>> >>> performance.readdir-ahead: on >>> >>> transport.address-family: inet >>> >>> features.uss: on >>> >>> >>> >>> Volume2.name: tv2 >>> >>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>> >>> Volume2.type: Distribute >>> >>> ......... >>> >>> ......... >>> >>> >>> >>> So essentially there would be a new section for every volume that would >>> >>> list the option names and corresponding values. Would adding this change >>> >>> still keep the get-state output parseable from Tendrl POV? >>> >>> >>> >>> Or would an output like the following make more sense? Let us know. >>> >>> Thanks. >>> >>> >>> >>> [Volumes] >>> >>> Volume1.name: tv1 >>> >>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>> >>> Volume1.type: Distribute >>> >>> >>> >>> Volume1.rebalance.files: 0 >>> >>> Volume1.rebalance.data: 0Bytes >>> >>> Volume1.options.nfs.disable: on >>> >>> Volume1.options.performance.readdir-ahead: on >>> >>> Volume1.options.transport.address-family: inet >>> >>> Volume1.options.features.uss: on >>> >>> >>> >>> Volume2.name: tv2 >>> >>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>> >>> Volume2.type: Distribute >>> >>> ......... >>> >>> ......... >>> >>> >>> >>> >>> >>> ~ Samikshan >>> >>> >>> >>> >>> >> >>> >> >>> > _______________________________________________ >>> > Tendrl-devel mailing list >>> > Tendrl-devel at redhat.com >>> > https://www.redhat.com/mailman/listinfo/tendrl-devel >>> > >>> >>> >>> >>> -- >>> >>> ~ Atin (atinm) >>> _______________________________________________ >>> Tendrl-devel mailing list >>> Tendrl-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/tendrl-devel >> >> > From jspray at redhat.com Tue Nov 22 15:33:13 2016 From: jspray at redhat.com (John Spray) Date: Tue, 22 Nov 2016 15:33:13 +0000 Subject: [Tendrl-devel] compute resources necessary for tendrl In-Reply-To: References: <2d76c8ee-177b-b82b-b317-320e893f1c37@redhat.com> Message-ID: On Tue, Nov 22, 2016 at 1:03 PM, Mrugesh Karnik wrote: > On 22 November 2016 at 08:45, Nishanth Thomas wrote: >> >> Here is what I am thinking about(in-line with what is proposed for > skyring ) >> >> * quad core CPU(Or multiple dual core CPUs) >> * 16 GB of RAM >> * One NIC with at-least 1 Gbps bandwidth >> >> thoughts? > > Now that we have a working build, I think we need to monitor and draw up > the actual numbers for resource utilisation by our various components. > Given that the components are more spread out, the original skyring based > numbers may not be fully valid. Pay particular attention to the IOPS consumed by your etcd cluster. If you are thinking of co-existing with the ceph mons (still haven't heard whether you are or not), this will be a key issue -- Ceph users do sometimes find that they saturate the local block device on a mon node. If you find that IO is a bottleneck, you could reduce your IO overhead by reading cluster maps straight out of ceph instead of taking copies of everything into etcd. John > > -- > Mrugesh > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From julim at redhat.com Tue Nov 22 16:13:52 2016 From: julim at redhat.com (Ju Lim) Date: Tue, 22 Nov 2016 11:13:52 -0500 Subject: [Tendrl-devel] Tendrl Hack Days Proposal (Week of 5 Dec 2016 in Bangalore, India) Message-ID: Team: Based on discussions during the Sprint 4 / Hackdays retrospective, one of the feedback was that we should try to do the hackdays 2 sprints before any planned release date. In conjunction, one of the suggestions was it may make sense to have another Hack Days event in Bangalore the week of 5 Dec 2016 to coincide with the Planning meetings that week, and both the Planning and Hack Days should be able to run in parallel. Below is a link to the upcoming Dec 2016 Hackdays event proposal, which lists the goal and logistics: https://docs.google.com/a/redhat.com/document/d/1NQ3FH90QoxPl_TCf2YevjkHuravHcOonr_ghDvCd11U/edit?usp=sharing Please take the time to take a look and add your comments and/or suggestions so we can get a rough agenda in place and firm up dates. Thank you, Ju From rkanade at redhat.com Wed Nov 23 07:51:19 2016 From: rkanade at redhat.com (Rohan Kanade) Date: Wed, 23 Nov 2016 13:21:19 +0530 Subject: [Tendrl-devel] Regarding some additional data points in the state output In-Reply-To: References: <5e1609e4-cfc3-4dc3-f421-ac51f56b9d64@redhat.com> <1988c0ba-6dfb-cdb3-a70e-4e51f02830e0@redhat.com> Message-ID: Tendrl Spec to track this change : https://github.com/Tendrl/specifications/pull/1 On Tue, Nov 22, 2016 at 8:16 PM, Rohan Kanade wrote: > Sample: > START>>> > > [Global] > MYUUID: 6bbf8ac2-22a0-4f08-b986-fe75aea9f654 > op-version: 40000 > > [Global options] > > [Peers] > > [Volumes] > Volume1.name: test-vol > Volume1.id: 7942d008-e300-4fd9-8af0-5a118afd8d3d > Volume1.type: Distribute > Volume1.transport_type: tcp > Volume1.status: Started > Volume1.brickcount: 1 > Volume1.Brick1.path: 172.17.0.2:/tmp/b1 > Volume1.Brick1.hostname: 172.17.0.2 > Volume1.Brick1.port: 49153 > Volume1.Brick1.rdma_port: 0 > Volume1.Brick1.status: Started > Volume1.Brick1.signedin: True > Volume1.snap_count: 0 > Volume1.stripe_count: 1 > Volume1.replica_count: 1 > Volume1.subvol_count: 1 > Volume1.arbiter_count: 0 > Volume1.disperse_count: 0 > Volume1.redundancy_count: 0 > Volume1.quorum_status: not_applicable > Volume1.snapd_svc.online_status: Offline > Volume1.snapd_svc.inited: True > Volume1.rebalance.id: 00000000-0000-0000-0000-000000000000 > Volume1.rebalance.status: not_started > Volume1.rebalance.failures: 0 > Volume1.rebalance.skipped: 0 > Volume1.rebalance.lookedup: 0 > Volume1.rebalance.files: 0 > Volume1.rebalance.data: 0Bytes > [Volume1.options] > features.barrier: on > transport.address-family: inet > performance.readdir-ahead: on > nfs.disable: on > > Volume2.name: test-vol1 > Volume2.id: 35854708-bb72-45a5-bdbd-77c51e5ebfb9 > Volume2.type: Distribute > Volume2.transport_type: tcp > Volume2.status: Started > Volume2.brickcount: 1 > Volume2.Brick1.path: 172.17.0.2:/tmp/b2 > Volume2.Brick1.hostname: 172.17.0.2 > Volume2.Brick1.port: 49152 > Volume2.Brick1.rdma_port: 0 > Volume2.Brick1.status: Started > Volume2.Brick1.signedin: True > Volume2.snap_count: 0 > Volume2.stripe_count: 1 > Volume2.replica_count: 1 > Volume2.subvol_count: 1 > Volume2.arbiter_count: 0 > Volume2.disperse_count: 0 > Volume2.redundancy_count: 0 > Volume2.quorum_status: not_applicable > Volume2.snapd_svc.online_status: Offline > Volume2.snapd_svc.inited: True > Volume2.rebalance.id: 00000000-0000-0000-0000-000000000000 > Volume2.rebalance.status: not_started > Volume2.rebalance.failures: 0 > Volume2.rebalance.skipped: 0 > Volume2.rebalance.lookedup: 0 > Volume2.rebalance.files: 0 > Volume2.rebalance.data: 0Bytes > [Volume2.options] > transport.address-family: inet > performance.readdir-ahead: on > nfs.disable: on > > > [Services] > svc1.name: glustershd > svc1.online_status: Offline > > svc2.name: nfs > svc2.online_status: Offline > > svc3.name: bitd > svc3.online_status: Offline > > svc4.name: scrub > svc4.online_status: Offline > > svc5.name: quotad > svc5.online_status: Offline > > > [Misc] > Base port: 49152 > Last allocated port: 49153 > > < > > On Tue, Nov 22, 2016 at 2:06 PM, Rohan Kanade wrote: > > > > Also, please provide a full state dump example with this patch included, > easier for Tendrl devs to get started without deploying this patch > > > > On Tue, Nov 22, 2016 at 1:44 PM, Rohan Kanade > wrote: > >> > >> Id prefer the first option > >> > >> > >> [Volumes] > >> Volume1.name: tv1 > >> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 > >> Volume1.type: Distribute > >> > >> Volume1.rebalance.files: 0 > >> Volume1.rebalance.data: 0Bytes > >> [Volume1.options] > >> nfs.disable: on > >> performance.readdir-ahead: on > >> transport.address-family: inet > >> features.uss: on > >> > >> Volume2.name: tv2 > >> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a > >> Volume2.type: Distribute > >> ......... > >> ......... > >> > >> > >> This would require minor changes to tendrl/gluster-integration > definition files and code. I will draw up a spec on Tendrl/specifications > to document the changes required. Please go ahead with your patch > >> Thanks > >> > >> On Tue, Nov 22, 2016 at 9:18 AM, Atin Mukherjee > wrote: > >>> > >>> We are awaiting final confirmation from Rohan. Samikshan has kept the > >>> changes ready and will push it to gerrit once we hear from Rohan. > >>> > >>> On Mon, Nov 21, 2016 at 12:54 PM, Shubhendu Tripathi < > shtripat at redhat.com> > >>> wrote: > >>> > >>> > Looking at options, I feel option-2 would be more feasible and might > not > >>> > need code changes in tendrl. > >>> > But still lets wait for the confirmation from Rohan. > >>> > > >>> > Regards, > >>> > Shubhendu > >>> > > >>> > > >>> > > >>> > On 11/21/2016 12:43 PM, Atin Mukherjee wrote: > >>> > > >>> >> +tendrl-devel > >>> >> > >>> >> On Mon, Nov 21, 2016 at 12:41 PM, Samikshan Bairagya < > sbairagy at redhat.com > >>> >> > > >>> >> wrote: > >>> >> > >>> >> Hey Rohan, > >>> >>> > >>> >>> So the current get-state CLI misses volume specific options in its > >>> >>> output. > >>> >>> Somehow I missed it while coming up with the implementation. This > patch > >>> >>> by > >>> >>> Atin is a fix for that: http://review.gluster.org/#/c/15889/1. The > >>> >>> following example shows how this patch would add these new data > points > >>> >>> and > >>> >>> how that would change the existing format: > >>> >>> > >>> >>> [Volumes] > >>> >>> Volume1.name: tv1 > >>> >>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 > >>> >>> Volume1.type: Distribute > >>> >>> > >>> >>> Volume1.rebalance.files: 0 > >>> >>> Volume1.rebalance.data: 0Bytes > >>> >>> [Volume1.options] > >>> >>> nfs.disable: on > >>> >>> performance.readdir-ahead: on > >>> >>> transport.address-family: inet > >>> >>> features.uss: on > >>> >>> > >>> >>> Volume2.name: tv2 > >>> >>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a > >>> >>> Volume2.type: Distribute > >>> >>> ......... > >>> >>> ......... > >>> >>> > >>> >>> So essentially there would be a new section for every volume that > would > >>> >>> list the option names and corresponding values. Would adding this > change > >>> >>> still keep the get-state output parseable from Tendrl POV? > >>> >>> > >>> >>> Or would an output like the following make more sense? Let us know. > >>> >>> Thanks. > >>> >>> > >>> >>> [Volumes] > >>> >>> Volume1.name: tv1 > >>> >>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 > >>> >>> Volume1.type: Distribute > >>> >>> > >>> >>> Volume1.rebalance.files: 0 > >>> >>> Volume1.rebalance.data: 0Bytes > >>> >>> Volume1.options.nfs.disable: on > >>> >>> Volume1.options.performance.readdir-ahead: on > >>> >>> Volume1.options.transport.address-family: inet > >>> >>> Volume1.options.features.uss: on > >>> >>> > >>> >>> Volume2.name: tv2 > >>> >>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a > >>> >>> Volume2.type: Distribute > >>> >>> ......... > >>> >>> ......... > >>> >>> > >>> >>> > >>> >>> ~ Samikshan > >>> >>> > >>> >>> > >>> >> > >>> >> > >>> > _______________________________________________ > >>> > Tendrl-devel mailing list > >>> > Tendrl-devel at redhat.com > >>> > https://www.redhat.com/mailman/listinfo/tendrl-devel > >>> > > >>> > >>> > >>> > >>> -- > >>> > >>> ~ Atin (atinm) > >>> _______________________________________________ > >>> Tendrl-devel mailing list > >>> Tendrl-devel at redhat.com > >>> https://www.redhat.com/mailman/listinfo/tendrl-devel > >> > >> > > > From rkanade at redhat.com Wed Nov 23 08:35:31 2016 From: rkanade at redhat.com (Rohan Kanade) Date: Wed, 23 Nov 2016 14:05:31 +0530 Subject: [Tendrl-devel] Tendrl Flow Definitions In-Reply-To: References: <885696456.190941.1479466632538.JavaMail.zimbra@redhat.com> Message-ID: Tendrl aims to provide multi cluster (ceph, gluster etc) management via a single API endpoint. We need to ensure the following things to enable that. - Node agents will always be cluster agnostic i.e. Node agents and their definition file will be outside the cluster namespace and will remain the same for that tendrl deployment, Hence we cannot aggregate the node agent definition with the cluster definitions - Any updates to the node agent definitions in etcd by monitoring or any other service can be discussed, these definitions under similar namespace can be aggregated . - The ceph and gluster integration have their own namespace which comes with a cluster context, any updates to this namespace can be aggregated. TL;DR: Multiple services (monitoring, node agent) can use one single path to store their definitions in etcd if and only if they belong to the same namespace. eg: tendrl.node_agent.objects and tendrl.node_agent.monitoring.objects Makes sense? On Fri, Nov 18, 2016 at 5:13 PM, Shubhendu Tripathi wrote: > I agree and this is something which we discussed yesterday in detail. > Request others to comment. > > Regards, > Shubhendu > > > > On 11/18/2016 04:27 PM, Anmol Babu wrote: > > Hi all, > > I have a suggestion for a very small enhancement in the current flow framework. > > Current working procedure: > > 1. Any service/agent defining and implementing a flow pushes the corresponding flow definitions to its specific definitions etcd directory > 2. The Agent/service flow framework reads its specific definition files and does all validations and executes the flow. > > ex: > 1. Node-agent pushes tendrl_definitions_node_agent to /tendrl_definitions_node_agent in etcd, gluster integration pushes tendrl_definitions_gluster_integration to /tendrl_definitions_gluster_integration and so on... > 2. node-agent is a service that reads flow definitions from /tendrl_definitions_node_agent in etcd and gluster-integration is a process that reads /tendrl_definitions_gluster_integration from etcd and currently each of them > replicates the flow definition parsing logic(probably its already currently thought through to move generic sections of flow yaml parsing to tendrl/common) > > Drawback: > > Any component requiring to define its own flows will then need to implement(currently followed)/extend(from a generic one in tendrl/common -- probably already thought of) the flow framework > and this requires the component to be run as a service as the flow framework in each of these component will only and always read only its specific definition and whenever there is a job(intended for it) > in the etcd queue. > > Suggestion: > > 1. Push all flow definitions to a path under a common root directory say for ex: /tendrl_definitons/ in etcd > 2. Now, when there's a job in the queue, the flow framework will make use of either: > i. A part of the full qualified package name in the run parameter of the job > or > ii. A separate field to indicate the definitions namespace(i.e, the name under the /tendrl_definitions/ where the definition can be obtained from) > to read definitions from appropriate etcd directory. > 3. And from there the already existing flow framework as usual will extract and execute the pre, post and the normal flow atoms. > > Relevance of the suggestion: > > 1. There is a central application called tendrl/performance_monitoring which performs the task of > i. Statistics aggregation > ii. Exposing time series db data > iii. Loading monitoring specific initial flow definitions to etcd > 2. There will be a node-side library that defines atoms and flows specific to monitoring. This is carved out from tendrl/performance_monitoring application(separate rpms). > The flow definition(yaml) being specific to monitoring, cannot be part of flow definitions(yaml) of any other project i.e, node_agent or even the gluster-integration or ceph-integration > because monitoring is intended to be completely outside core stack. Now, given this, consider the case of a monitoring flow to generate collectd configurations on the nodes. In such a case > the idea is to employ a python script(wrapper around jinja2 templating) that is executed as a command by the flow > (This is not still generic as internally the command is not just a mere jinja2 wrapper but contains some specifics of collectd). > So that the flow is very generic(with very specific parameters). But exposing > this as a flow to execute any normal command can turn out to be a breach of security and hence the command needs to be hard-coded in the flow's respective atom which makes it still monitoring specific > and hence needs to be part of an application that is specific to monitoring. But, with the current flow framework, just so that flow is executed, the flow implementer needs to be a service. > But, with the small change suggested above the implementing application can be a mere stateless library(like tendrl/common) and any and every flow execution can be streamlined to be executed by the node-agent > by just a simple python import of the specific installed python source which can be provided by the library(specific tendrl project). > > > Please consider this idea and provide your valuable suggestions/feedback. > > > Thanks, > Anmol > > > From rkanade at redhat.com Wed Nov 23 11:36:08 2016 From: rkanade at redhat.com (Rohan Kanade) Date: Wed, 23 Nov 2016 17:06:08 +0530 Subject: [Tendrl-devel] [Release] Tendrl v1.1 Message-ID: We are pleased to announce the release of Tendrl v1.1 This release mainly provides RPM specs for each Tendrl component Please refer to individual Tendrl components for more details about this release: Tendrl-common: https://github.com/Tendrl/common/releases/tag/v1.1 Tendrl-node-agent: https://github.com/Tendrl/node_agent/releases/tag/v1.1 Tendrl-ceph-integration: https://github.com/Tendrl/ceph_integration/releases/tag/v1.1 Tendrl-gluster-integration: https://github.com/Tendrl/gluster_integration/releases/tag/v1.1 Tendrl-API: https://github.com/Tendrl/tendrl-api/releases/tag/v1.1 Thank you to all Tendrl contributors. Regards, Rohan Kanade From nthomas at redhat.com Wed Nov 23 13:19:24 2016 From: nthomas at redhat.com (Nishanth Thomas) Date: Wed, 23 Nov 2016 18:49:24 +0530 Subject: [Tendrl-devel] Tendrl Packages Available Message-ID: All, We are pleased to announce the availability of RPM packages. Repository : https://copr.fedorainfracloud.org/coprs/tendrl/tendrl/ Packages : https://copr.fedorainfracloud.org/coprs/tendrl/tendrl/packages/ Repo file can be downloaded(For example- centos epel --> https://copr.fedorainfracloud.org/coprs/tendrl/tendrl/repo/epel-7/tendrl-tendrl-epel-7.repo ) from the repository. Thanks, Nishanth From nthomas at redhat.com Wed Nov 23 13:21:24 2016 From: nthomas at redhat.com (Nishanth Thomas) Date: Wed, 23 Nov 2016 18:51:24 +0530 Subject: [Tendrl-devel] Fwd: Recorded video of using tendrl with packages In-Reply-To: <1226237418.953706.1479903485520.JavaMail.zimbra@redhat.com> References: <2121832196.952305.1479903114741.JavaMail.zimbra@redhat.com> <1226237418.953706.1479903485520.JavaMail.zimbra@redhat.com> Message-ID: ---------- Forwarded message ---------- From: Darshan Narayana Murthy Date: Wed, Nov 23, 2016 at 5:48 PM Subject: Recorded video of using tendrl with packages To: Nishanth Thomas Hi Nishanth, While I did the last verification run of the gluster intigration package, I recorded that in bluejeans so that it can be used by people who want try out tendrl using packages. It contains basic package installation, basic configuration, import-gluster-cluster(api), create/delete volume (api). The link for recording: https://bluejeans.com/s/pq_kA/ you can share it if you feel its helpful -Darshan From kdreyer at redhat.com Wed Nov 23 13:44:56 2016 From: kdreyer at redhat.com (Ken Dreyer) Date: Wed, 23 Nov 2016 06:44:56 -0700 Subject: [Tendrl-devel] Tendrl Packages Available In-Reply-To: References: Message-ID: Hi Nishanth, This is great, thanks. I see version numbers like "0.0.1" here: tendrl-api-0.0.1-1.el7 tendrl-ceph-integration-0.0.1-1.el7 tendrl-common-0.0.1-1.el7 tendrl-gluster-integration-0.0.1-1.el7 tendrl-node-agent-0.0.1-1.el7 But Rohan is announcing "v1.1" Git tags in a separate thread? Would you please align these version numbers with the Git tags? - Ken On Wed, Nov 23, 2016 at 6:19 AM, Nishanth Thomas wrote: > All, > > We are pleased to announce the availability of RPM packages. > > Repository : https://copr.fedorainfracloud.org/coprs/tendrl/tendrl/ > Packages : https://copr.fedorainfracloud.org/coprs/tendrl/tendrl/packages/ > > Repo file can be downloaded(For example- centos epel --> > https://copr.fedorainfracloud.org/coprs/tendrl/tendrl/repo/epel-7/tendrl-tendrl-epel-7.repo > ) from the repository. > > Thanks, > Nishanth > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From nthomas at redhat.com Wed Nov 23 13:53:06 2016 From: nthomas at redhat.com (Nishanth Thomas) Date: Wed, 23 Nov 2016 19:23:06 +0530 Subject: [Tendrl-devel] Tendrl Packages Available In-Reply-To: References: Message-ID: Sure will do that soon On Wed, Nov 23, 2016 at 7:14 PM, Ken Dreyer wrote: > Hi Nishanth, > > This is great, thanks. > > I see version numbers like "0.0.1" here: > > tendrl-api-0.0.1-1.el7 > tendrl-ceph-integration-0.0.1-1.el7 > tendrl-common-0.0.1-1.el7 > tendrl-gluster-integration-0.0.1-1.el7 > tendrl-node-agent-0.0.1-1.el7 > > But Rohan is announcing "v1.1" Git tags in a separate thread? > > Would you please align these version numbers with the Git tags? > > - Ken > > > On Wed, Nov 23, 2016 at 6:19 AM, Nishanth Thomas > wrote: > > All, > > > > We are pleased to announce the availability of RPM packages. > > > > Repository : https://copr.fedorainfracloud.org/coprs/tendrl/tendrl/ > > Packages : https://copr.fedorainfracloud.org/coprs/tendrl/tendrl/ > packages/ > > > > Repo file can be downloaded(For example- centos epel --> > > https://copr.fedorainfracloud.org/coprs/tendrl/tendrl/repo/ > epel-7/tendrl-tendrl-epel-7.repo > > ) from the repository. > > > > Thanks, > > Nishanth > > _______________________________________________ > > Tendrl-devel mailing list > > Tendrl-devel at redhat.com > > https://www.redhat.com/mailman/listinfo/tendrl-devel > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From anivargi at redhat.com Wed Nov 23 14:49:51 2016 From: anivargi at redhat.com (Anup Nivargi) Date: Wed, 23 Nov 2016 20:19:51 +0530 Subject: [Tendrl-devel] [Release] Tendrl UI 1.0 upstream release Message-ID: <03B93B5C-D43A-492A-9A40-5FF14E2E57A7@redhat.com> Hi, Tendrl UI v1.0 has been released upstream [1] The release includes initial framework setup and basic storage management operations like importing a Ceph/Gluster cluster, creating/deleting Ceph pools and creating/deleting Gluster volumes. I would like to thank everyone who have contributed to the release. [1] https://github.com/Tendrl/tendrl_frontend/releases/tag/v1.0 -- Anup Nivargi From kdreyer at redhat.com Wed Nov 23 14:53:19 2016 From: kdreyer at redhat.com (Ken Dreyer) Date: Wed, 23 Nov 2016 07:53:19 -0700 Subject: [Tendrl-devel] [Release] Tendrl v1.1 In-Reply-To: References: Message-ID: On Wed, Nov 23, 2016 at 4:36 AM, Rohan Kanade wrote: > Tendrl-ceph-integration: > https://github.com/Tendrl/ceph_integration/releases/tag/v1.1 This repository still has all the old calamari tags. This makes "git describe origin/master" show the wrong version for this repo (v1.3.3-78-g021be63) Would you please delete the old Calamari tags from this repo, so "git describe" works as expected? Making "git describe" work properly is a key part of having Jenkins automatically build packages from Git snapshots. - Ken From mcarrano at redhat.com Wed Nov 23 20:30:35 2016 From: mcarrano at redhat.com (Matt Carrano) Date: Wed, 23 Nov 2016 15:30:35 -0500 Subject: [Tendrl-devel] Tendrl UX designs for review Message-ID: Three new UX designs are published for review. Please see the links below. Rebalance File Share (TEN-163): https://redhat.invisionapp.com/share/AB94BNET6 Create Ceph Cluster (TEN-164): https://redhat.invisionapp.com/share/2K8M4PQYZ Delete File Share (TEN-165): https://redhat.invisionapp.com/share/729GRP1W9 Comments may be left directly in the InVision documents. If you have never commented in InVision before, you should read this article https://support.invisionapp.com/hc/en-us/articles/209192426-How-do-I-comment-on-a-prototype- Please review and comment by COB on Wed Nov 30. We will be scheduling a follow-up review meeting sometime in the next 2 weeks. Regards, Matt -- Matt Carrano Sr. Interaction Designer Red Hat, Inc. mcarrano at redhat.com From rkanade at redhat.com Thu Nov 24 08:43:10 2016 From: rkanade at redhat.com (Rohan Kanade) Date: Thu, 24 Nov 2016 14:13:10 +0530 Subject: [Tendrl-devel] [Release] Tendrl v1.1 In-Reply-To: References: Message-ID: Ken, Deleted old tags, Thanks On Wed, Nov 23, 2016 at 8:23 PM, Ken Dreyer wrote: > On Wed, Nov 23, 2016 at 4:36 AM, Rohan Kanade wrote: > > Tendrl-ceph-integration: > > https://github.com/Tendrl/ceph_integration/releases/tag/v1.1 > > This repository still has all the old calamari tags. This makes "git > describe origin/master" show the wrong version for this repo > (v1.3.3-78-g021be63) > > Would you please delete the old Calamari tags from this repo, so "git > describe" works as expected? > > Making "git describe" work properly is a key part of having Jenkins > automatically build packages from Git snapshots. > > > - Ken > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From mbukatov at redhat.com Thu Nov 24 09:44:36 2016 From: mbukatov at redhat.com (Martin Bukatovic) Date: Thu, 24 Nov 2016 10:44:36 +0100 Subject: [Tendrl-devel] settings of usmqe repositories Message-ID: <087c8c76-7f9f-1b85-eb5c-3e0531120473@redhat.com> Hi Jeff, to be able to setup a webook for sphinx documentation builds, I need access to "Settings" page of usmqe repositories in Tendrl organization on github. Could you change the access rights for me to make this possible? Thank you -- Martin Bukatovic USM QE team From mrugesh at brainfunked.org Thu Nov 24 11:06:42 2016 From: mrugesh at brainfunked.org (Mrugesh Karnik) Date: Thu, 24 Nov 2016 16:36:42 +0530 Subject: [Tendrl-devel] settings of usmqe repositories In-Reply-To: <087c8c76-7f9f-1b85-eb5c-3e0531120473@redhat.com> References: <087c8c76-7f9f-1b85-eb5c-3e0531120473@redhat.com> Message-ID: Have granted the QE team admin access to the two usmqe repositories. On 24 November 2016 at 15:14, Martin Bukatovic wrote: > Hi Jeff, > > to be able to setup a webook for sphinx documentation builds, > I need access to "Settings" page of usmqe repositories in Tendrl > organization on github. Could you change the access rights for > me to make this possible? > > Thank you > > -- > Martin Bukatovic > USM QE team > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From mrugesh at brainfunked.org Thu Nov 24 11:38:30 2016 From: mrugesh at brainfunked.org (Mrugesh Karnik) Date: Thu, 24 Nov 2016 17:08:30 +0530 Subject: [Tendrl-devel] Update to the jira & github usage and the development workflow Message-ID: IMPORTANT: It'll henceforth be mandatory that every pull request be against either a github issue or a specification (more on this below) and have complete unit test coverage. The specifications repository[1] itself will make use of github issues and pull requests. The only exception being that in the specifications repository, the pull requests need not reference an existing issue or specification. Details follow.. The usage of the tools such as jira and github has been under discussion a few times on this list. This is an updated method of tracking and organising the project, based on the observations from the development cycle of the first release. Key observations: 1. Developers are extremely comfortable with following the github workflow of issues and pull requests for development. 2. Compared to github issues, jira feels rather out-of-band for developers. This is especially the case during periods of high code churn. 3. Github issues, while more comfortable for developers, does not provide a project-wide view to be able to track the project as a whole. Jira feels like a better tool for such information. This updated workflow tries to clearly separate out the domains of jira and github tools. Jira: * Jira would be used for driving the project development based on an end-user perspective. * Feature planning for the releases would be done in jira. * User stories would be populated in jira. Github: * Github would be used for any development activity. * Features and user stories discussed and approved in jira would be brought to github as implementation specifications. * Developers would work based on github issues and pull requests. * Tracking and prioritising of the issues on github will be done using a combination of the labels, milestones and projects. Specifications: `Specifications' are a restructuring of the existing workflow that uses jira epics and the documentation repository (not necessarily together or in-sync). Specifications will be the single point of origin for any enhancements (includes features and user stories) to be implemented in the project codebases. A new repository has been created to host the specifications in. This repository, like all others, will be worked on via pull requests. Rohan, Nishanth and I, currently, will be looking after the specifications (referred to as the `specification maintainers' or `SMs' here onwards). This includes reviews for any contributed specifications, conversion of mailing list and jira discussions into specifications etc. Specifications would provide project-wide view of enhancements. This allows for the impact on various sub-projects such as the framework, storage system integrations, add-on stacks and components etc. to be scoped out. >From specifications, issues would be created in the sub-project repositories for implementation. The Proposed Workflows: Feature proposals: * Propose an enhancement via discussion on the mailing list, irc or jira[1]. * Send a pull request on the specifications repository with the details for implementation. * The SMs will ensure that the specifications map to a corresponding feature or user story in jira. The entry would be created if one doesn't exist. Discussions on jira would allow the specifications to be prioritised for implementation. General development: * Create (or help the SMs create) github issues on the individual repositories based on the specifications. * Report bugs as github issues directly. * Send pull requests against the existing github issues with the issue and/or the specification id clearly mentioned in the commit message. [1] https://github.com/Tendrl/specifications -- Mrugesh From deb at redhat.com Thu Nov 24 12:26:58 2016 From: deb at redhat.com (Soumya Deb) Date: Thu, 24 Nov 2016 17:56:58 +0530 Subject: [Tendrl-devel] Update to the jira & github usage and the development workflow In-Reply-To: References: Message-ID: I agree, developers are more accustomed to and comfortable using github for their workflow, but if we're going ahead with this spec, then I think having a GitHub-Jira bridge (to create/replicate/reply etc) is pretty much necessary at the same time of the spec going in effect. Without that the information disconnect would be too huge for those two systems to stay relevant (more so, for the users of each tool). On 24 November 2016 at 17:08, Mrugesh Karnik wrote: > IMPORTANT: > > It'll henceforth be mandatory that every pull request be against either a > github issue or a specification (more on this below) and have complete unit > test coverage. > > The specifications repository[1] itself will make use of github issues and > pull requests. The only exception being that in the > specifications repository, the pull requests need not reference an existing > issue or specification. > > Details follow.. > > > > The usage of the tools such as jira and github has been under discussion a > few times on this list. This is an updated method of tracking and > organising the project, based on the observations from the development > cycle of the first release. > > Key observations: > 1. Developers are extremely comfortable with following the github workflow > of issues and pull requests for development. > 2. Compared to github issues, jira feels rather out-of-band for developers. > This is especially the case during periods of high code churn. > 3. Github issues, while more comfortable for developers, does not provide a > project-wide view to be able to track the project as a whole. Jira feels > like a better tool for such information. > > > This updated workflow tries to clearly separate out the domains of jira and > github tools. > > Jira: > > * Jira would be used for driving the project development based on > an end-user perspective. > * Feature planning for the releases would be done in jira. > * User stories would be populated in jira. > > Github: > > * Github would be used for any development activity. > * Features and user stories discussed and approved in jira would be brought > to github as implementation specifications. > * Developers would work based on github issues and pull requests. > * Tracking and prioritising of the issues on github will be done using a > combination of the labels, milestones and projects. > > > Specifications: > > `Specifications' are a restructuring of the existing workflow that uses > jira epics and the documentation repository (not necessarily together or > in-sync). Specifications will be the single point of origin for any > enhancements (includes features and user stories) to be implemented in the > project codebases. A new repository has been > created to host the specifications in. This repository, like all others, > will be worked on via pull requests. Rohan, Nishanth and I, currently, will > be looking after the specifications (referred to as the `specification > maintainers' or `SMs' here onwards). This includes reviews for any > contributed specifications, conversion of mailing list > and jira discussions into specifications etc. > > Specifications would provide project-wide view of enhancements. This allows > for the impact on various sub-projects such as the framework, storage > system integrations, add-on stacks and components etc. to be scoped out. > >From specifications, issues would be created in the sub-project > repositories for implementation. > > > The Proposed Workflows: > > Feature proposals: > * Propose an enhancement via discussion on the mailing list, irc or > jira[1]. > * Send a pull request on the specifications repository with the details for > implementation. > * The SMs will ensure that the specifications map to a > corresponding feature or user story in jira. The entry would be created if > one doesn't exist. Discussions on jira would allow the specifications to be > prioritised for implementation. > > General development: > * Create (or help the SMs create) github issues on the > individual repositories based on the specifications. > * Report bugs as github issues directly. > * Send pull requests against the existing github issues with the issue > and/or the specification id clearly mentioned in the commit message. > > > [1] https://github.com/Tendrl/specifications > > -- > Mrugesh > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From mbukatov at redhat.com Thu Nov 24 12:41:29 2016 From: mbukatov at redhat.com (Martin Bukatovic) Date: Thu, 24 Nov 2016 13:41:29 +0100 Subject: [Tendrl-devel] settings of usmqe repositories In-Reply-To: References: <087c8c76-7f9f-1b85-eb5c-3e0531120473@redhat.com> Message-ID: <78ac94d6-f280-7da3-a491-ceee7cae6ce0@redhat.com> On 11/24/2016 12:06 PM, Mrugesh Karnik wrote: > Have granted the QE team admin access to the two usmqe repositories. Thank you! The hook has been configured and QE documentation is now available at: https://usmqe-tests.readthedocs.io/en/latest/ -- Martin Bukatovic USM QE team From japplewh at redhat.com Thu Nov 24 16:38:05 2016 From: japplewh at redhat.com (Jeff Applewhite) Date: Thu, 24 Nov 2016 11:38:05 -0500 Subject: [Tendrl-devel] settings of usmqe repositories In-Reply-To: <78ac94d6-f280-7da3-a491-ceee7cae6ce0@redhat.com> References: <087c8c76-7f9f-1b85-eb5c-3e0531120473@redhat.com> <78ac94d6-f280-7da3-a491-ceee7cae6ce0@redhat.com> Message-ID: Thanks all I'll be eating Turkey today ;-) But I love to see the progress in automated testing! We will be moving quickly in the very near future with features (and needed framework changes possibly) and so this automation is key to our ability to do so quickly without regressions. Looking forward to a demo of out capabilities in the near future. Best Jeff On Thu, Nov 24, 2016 at 7:41 AM, Martin Bukatovic wrote: > On 11/24/2016 12:06 PM, Mrugesh Karnik wrote: > > Have granted the QE team admin access to the two usmqe repositories. > > Thank you! The hook has been configured and QE documentation is now > available at: > > https://usmqe-tests.readthedocs.io/en/latest/ > > -- > Martin Bukatovic > USM QE team > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > -- Jeff Applewhite Principal Product Manager From mbukatov at redhat.com Thu Nov 24 17:04:59 2016 From: mbukatov at redhat.com (Martin Bukatovic) Date: Thu, 24 Nov 2016 18:04:59 +0100 Subject: [Tendrl-devel] Tendrl UX designs for review In-Reply-To: References: Message-ID: On 11/23/2016 09:30 PM, Matt Carrano wrote: > Three new UX designs are published for review. Please see the links below. > > Rebalance File Share (TEN-163): > https://redhat.invisionapp.com/share/AB94BNET6 > > Create Ceph Cluster (TEN-164): > https://redhat.invisionapp.com/share/2K8M4PQYZ > > Delete File Share (TEN-165): https://redhat.invisionapp.com/share/729GRP1W9 > > Comments may be left directly in the InVision documents. If you have never > commented in InVision before, you should read this article > https://support.invisionapp.com/hc/en-us/articles/209192426-How-do-I-comment-on-a-prototype- > > Please review and comment by COB on Wed Nov 30. We will be scheduling a > follow-up review meeting sometime in the next 2 weeks. I don't see these new designs references in ui repository[1], did we stop using it? I think it's very valuable to have a single index of all designs maintained somewhere. [1] https://github.com/Tendrl/ui/blob/master/UI%20Designs.adoc -- Martin Bukatovic USM QE team From julim at redhat.com Thu Nov 24 17:29:28 2016 From: julim at redhat.com (Ju Lim) Date: Thu, 24 Nov 2016 12:29:28 -0500 Subject: [Tendrl-devel] Tendrl UX designs for review In-Reply-To: References: Message-ID: We just haven't yet updated the doc but the plan is to. On Nov 24, 2016 12:05 PM, "Martin Bukatovic" wrote: > On 11/23/2016 09:30 PM, Matt Carrano wrote: > > Three new UX designs are published for review. Please see the links > below. > > > > Rebalance File Share (TEN-163): > > https://redhat.invisionapp.com/share/AB94BNET6 > > > > Create Ceph Cluster (TEN-164): > > https://redhat.invisionapp.com/share/2K8M4PQYZ > > > > Delete File Share (TEN-165): https://redhat.invisionapp. > com/share/729GRP1W9 > > > > Comments may be left directly in the InVision documents. If you have > never > > commented in InVision before, you should read this article > > https://support.invisionapp.com/hc/en-us/articles/ > 209192426-How-do-I-comment-on-a-prototype- > > > > Please review and comment by COB on Wed Nov 30. We will be scheduling a > > follow-up review meeting sometime in the next 2 weeks. > > I don't see these new designs references in ui repository[1], > did we stop using it? I think it's very valuable to have a > single index of all designs maintained somewhere. > > [1] https://github.com/Tendrl/ui/blob/master/UI%20Designs.adoc > > -- > Martin Bukatovic > USM QE team > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From fbalak at redhat.com Mon Nov 28 12:48:50 2016 From: fbalak at redhat.com (Filip Balak) Date: Mon, 28 Nov 2016 07:48:50 -0500 (EST) Subject: [Tendrl-devel] Packaging issues In-Reply-To: <1093119030.176788.1480337228754.JavaMail.zimbra@redhat.com> Message-ID: <818316411.177915.1480337330705.JavaMail.zimbra@redhat.com> Hi all, where should I record issues concerning packaging? (e.g. EPEL dependency) Best Regards, Filip Bal?k From sankarshan.mukhopadhyay at gmail.com Mon Nov 28 12:52:08 2016 From: sankarshan.mukhopadhyay at gmail.com (Sankarshan Mukhopadhyay) Date: Mon, 28 Nov 2016 18:22:08 +0530 Subject: [Tendrl-devel] Packaging issues In-Reply-To: <818316411.177915.1480337330705.JavaMail.zimbra@redhat.com> References: <1093119030.176788.1480337228754.JavaMail.zimbra@redhat.com> <818316411.177915.1480337330705.JavaMail.zimbra@redhat.com> Message-ID: On Mon, Nov 28, 2016 at 6:18 PM, Filip Balak wrote: > where should I record issues concerning packaging? (e.g. EPEL dependency) [I took out what seems to be a non-public mailing list]. Would it be possible to provide more detail? On the face of it, looks like an issue which should be filed against the component/repo on Tendrl -- sankarshan mukhopadhyay From mbukatov at redhat.com Mon Nov 28 13:07:31 2016 From: mbukatov at redhat.com (Martin Bukatovic) Date: Mon, 28 Nov 2016 14:07:31 +0100 Subject: [Tendrl-devel] Packaging issues In-Reply-To: References: <1093119030.176788.1480337228754.JavaMail.zimbra@redhat.com> <818316411.177915.1480337330705.JavaMail.zimbra@redhat.com> Message-ID: On 11/28/2016 01:52 PM, Sankarshan Mukhopadhyay wrote: > Would it be > possible to provide more detail? On the face of it, looks like an > issue which should be filed against the component/repo on Tendrl There are 2 questions here: * Wow to track upstream packaging issues? * Is epel dependency of some tendrl package ok? At first sight, we don't think so, and we would like to track and discuss this in the issue properly. So let's focus on the 1st question in this mailing list only. We could discuss the 2nd when we create an issue/ticket. My original understanding was that packaging issues should go to the Tendrl JIRA, because the packaging isn't usually is directly done in upstream git repository. While some spec file can be committed in the repo upstream, it doesn't mean it is used during whatever build we have in the copr or koji. That said, since there are some changes in the process, I would like to get an agreement where should the packaging issues be reported. You are suggesting to use github repo of the affected component. Is that ok with everyone? As far as QE team is concerned, we don't care, but we would like to report those issues in a way which all tendrl devs would agree with. -- Martin Bukatovic USM QE team From sankarshan.mukhopadhyay at gmail.com Mon Nov 28 13:09:59 2016 From: sankarshan.mukhopadhyay at gmail.com (Sankarshan Mukhopadhyay) Date: Mon, 28 Nov 2016 18:39:59 +0530 Subject: [Tendrl-devel] Packaging issues In-Reply-To: References: <1093119030.176788.1480337228754.JavaMail.zimbra@redhat.com> <818316411.177915.1480337330705.JavaMail.zimbra@redhat.com> Message-ID: On Mon, Nov 28, 2016 at 6:37 PM, Martin Bukatovic wrote: > There are 2 questions here: > > * Wow to track upstream packaging issues? > * Is epel dependency of some tendrl package ok? At first sight, > we don't think so, and we would like to track and discuss this > in the issue properly. To be fair, without adequate detail about the issue, conversations would be in hypotheticals. Could we at least understand what 'dependency' means in this context? -- sankarshan mukhopadhyay From mbukatov at redhat.com Mon Nov 28 13:16:14 2016 From: mbukatov at redhat.com (Martin Bukatovic) Date: Mon, 28 Nov 2016 14:16:14 +0100 Subject: [Tendrl-devel] Packaging issues In-Reply-To: References: <1093119030.176788.1480337228754.JavaMail.zimbra@redhat.com> <818316411.177915.1480337330705.JavaMail.zimbra@redhat.com> Message-ID: <5a7fd0c5-9719-a99c-024a-1ca4ebf58bcd@redhat.com> On 11/28/2016 02:09 PM, Sankarshan Mukhopadhyay wrote: > To be fair, without adequate detail about the issue, conversations > would be in hypotheticals. Could we at least understand what > 'dependency' means in this context? We will provide all the necessary details in the ticket/issue, when we have an agreement on where to actually report it. By packaging issue, I mean anything which is related to the specfile code, build root, rpm dependencies, and so on. -- Martin Bukatovic USM QE team From rkanade at redhat.com Mon Nov 28 13:30:04 2016 From: rkanade at redhat.com (Rohan Kanade) Date: Mon, 28 Nov 2016 19:00:04 +0530 Subject: [Tendrl-devel] Packaging issues In-Reply-To: <5a7fd0c5-9719-a99c-024a-1ca4ebf58bcd@redhat.com> References: <1093119030.176788.1480337228754.JavaMail.zimbra@redhat.com> <818316411.177915.1480337330705.JavaMail.zimbra@redhat.com> <5a7fd0c5-9719-a99c-024a-1ca4ebf58bcd@redhat.com> Message-ID: Martin, We use github issues [0] to track anything related to Tendrl. Please file an issue on the Tendrl component repo for which you are facing dependency problems. [0]: https://www.redhat.com/archives/tendrl-devel/2016-November/msg00101.html On Mon, Nov 28, 2016 at 6:46 PM, Martin Bukatovic wrote: > On 11/28/2016 02:09 PM, Sankarshan Mukhopadhyay wrote: > > To be fair, without adequate detail about the issue, conversations > > would be in hypotheticals. Could we at least understand what > > 'dependency' means in this context? > > We will provide all the necessary details in the ticket/issue, when > we have an agreement on where to actually report it. By packaging > issue, I mean anything which is related to the specfile code, build > root, rpm dependencies, and so on. > > -- > Martin Bukatovic > USM QE team > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From mbukatov at redhat.com Mon Nov 28 13:45:12 2016 From: mbukatov at redhat.com (Martin Bukatovic) Date: Mon, 28 Nov 2016 14:45:12 +0100 Subject: [Tendrl-devel] Packaging issues In-Reply-To: References: <1093119030.176788.1480337228754.JavaMail.zimbra@redhat.com> <818316411.177915.1480337330705.JavaMail.zimbra@redhat.com> <5a7fd0c5-9719-a99c-024a-1ca4ebf58bcd@redhat.com> Message-ID: <9c2c044a-da6b-e1dc-09bb-2ddc5e5d73c0@redhat.com> On 11/28/2016 02:30 PM, Rohan Kanade wrote: > We use github issues [0] to track anything related to Tendrl. > > Please file an issue on the Tendrl component repo for which you are facing > dependency problems. Ok, thanks for the confirmation. -- Martin Bukatovic USM QE team From fulton at redhat.com Mon Nov 28 14:38:50 2016 From: fulton at redhat.com (John Fulton) Date: Mon, 28 Nov 2016 09:38:50 -0500 (EST) Subject: [Tendrl-devel] compute resources necessary for tendrl In-Reply-To: References: <2d76c8ee-177b-b82b-b317-320e893f1c37@redhat.com> Message-ID: <541522066.885615.1480343930756.JavaMail.zimbra@redhat.com> Inline... ----- Original Message ----- From: "John Spray" To: "Mailing list for the contributors to the Tendrl project" Sent: Tuesday, November 22, 2016 10:33:13 AM Subject: Re: [Tendrl-devel] compute resources necessary for tendrl > On Tue, Nov 22, 2016 at 1:03 PM, Mrugesh Karnik wrote: > > On 22 November 2016 at 08:45, Nishanth Thomas wrote: > >> > >> Here is what I am thinking about(in-line with what is proposed for > > skyring ) > >> > >> * quad core CPU(Or multiple dual core CPUs) > >> * 16 GB of RAM > >> * One NIC with at-least 1 Gbps bandwidth > >> > >> thoughts? > > > > Now that we have a working build, I think we need to monitor and draw up > > the actual numbers for resource utilisation by our various components. > > Given that the components are more spread out, the original skyring based > > numbers may not be fully valid. > > Pay particular attention to the IOPS consumed by your etcd cluster. > If you are thinking of co-existing with the ceph mons (still haven't > heard whether you are or not), this will be a key issue -- Ceph users > do sometimes find that they saturate the local block device on a mon > node. > > If you find that IO is a bottleneck, you could reduce your IO overhead > by reading cluster maps straight out of ceph instead of taking copies > of everything into etcd. > > John Thanks for the follow up here. The conversation has steered towards production type requirements, which is useful but I wanted to add the outcome of an out-of-band conversation I had with mkarnik. For the goal of just verifying functionality for something like CI, mkarnik said he was able to get away with the following for a VM to run Tendrl: 1 vCPU Core 1 Gig RAM 9 Gig disk space Again, I want to emphasize the above is for minimum functionality verification. I'll be using these values in my testing in my development environment for TripleO integration [1]. Thanks, John [1] https://review.openstack.org/#/c/387631/13/specs/ocata/tripleo-tendrl-integration.rst From julim at redhat.com Mon Nov 28 16:34:38 2016 From: julim at redhat.com (Ju Lim) Date: Mon, 28 Nov 2016 11:34:38 -0500 Subject: [Tendrl-devel] Notification strats [TEN-137] and UXD vs Dynamic UI Message-ID: Team: For folks who may have missed the discussion earlier on Notification strats [TEN-137 Comparison of websocket, server push mechanism and continuous polling ] and UXD vs Dynamic UI, here's the recording: https://bluejeans.com/s/Cqtnf/ Presentation shared related to TEN-137: https://tendrl.atlassian.net/secure/attachment/10200/Polling%20vs%20WS%20vs%20SSE.pdf Regards, Ju From julim at redhat.com Mon Nov 28 21:05:17 2016 From: julim at redhat.com (Ju Lim) Date: Mon, 28 Nov 2016 16:05:17 -0500 Subject: [Tendrl-devel] Sprint 5 Demo tomorrow Message-ID: Team: Tomorrow we have the Sprint 5 demo scheduled. Please take a moment to review the Sprint 5 demo agenda and update as needed. The goal of the demos is to show work that has met the Definition of Done (DoD) ; however, if we don't have anything yet meeting the DoD, we would still like to demo/review work that has been completed (code complete) and also to better understand the progress the team is making. I have conflicting commitments tomorrow through Thursday, Dec 1 and will try to attend the Tendrl meetings scheduled but likely won't be able to. Hence, Jeff A. and Sharmilla will be filling in for me in my absence. Jeff & Sharmilla - please record the Sprint demo (Tues, Nov 29) and Sprint 6 planning (Wed, Nov 30). If folks have any questions and/or concerns, please email me off list. Thanks, Ju From rkanade at redhat.com Tue Nov 29 09:19:17 2016 From: rkanade at redhat.com (Rohan Kanade) Date: Tue, 29 Nov 2016 14:49:17 +0530 Subject: [Tendrl-devel] Update to the jira & github usage and the development workflow In-Reply-To: References: Message-ID: The following references will be necessary for all Tendrl git commits moving on Git commit msgs: - Must contain reference to github issue being fixed, "tendrl-bug-id: Tendrl/#" on a new line in the commit msg - Optionally can contain reference to tendrl specification being implemented, "tendrl-spec: " on a new line in the commit msg (https://github.com/Tendrl/specifications/tree/master/specs) Example: - https://github.com/Tendrl/node_agent/pull/77 - https://github.com/Tendrl/node_agent/issues/73 Repos implementing above changes: - Tendrl/common, Tendrl/node_agent, Tendrl/gluster_integration, Tendrl/ceph_integration, Tendrl/performance_monitoring, Tendrl/alerting I request other Tendrl repo owners to also implement these changes (Tendrl/tendrl-api, Tendrl/tendrl_frontend) On Thu, Nov 24, 2016 at 5:08 PM, Mrugesh Karnik wrote: > IMPORTANT: > > It'll henceforth be mandatory that every pull request be against either a > github issue or a specification (more on this below) and have complete unit > test coverage. > > The specifications repository[1] itself will make use of github issues and > pull requests. The only exception being that in the > specifications repository, the pull requests need not reference an existing > issue or specification. > > Details follow.. > > > > The usage of the tools such as jira and github has been under discussion a > few times on this list. This is an updated method of tracking and > organising the project, based on the observations from the development > cycle of the first release. > > Key observations: > 1. Developers are extremely comfortable with following the github workflow > of issues and pull requests for development. > 2. Compared to github issues, jira feels rather out-of-band for developers. > This is especially the case during periods of high code churn. > 3. Github issues, while more comfortable for developers, does not provide a > project-wide view to be able to track the project as a whole. Jira feels > like a better tool for such information. > > > This updated workflow tries to clearly separate out the domains of jira and > github tools. > > Jira: > > * Jira would be used for driving the project development based on > an end-user perspective. > * Feature planning for the releases would be done in jira. > * User stories would be populated in jira. > > Github: > > * Github would be used for any development activity. > * Features and user stories discussed and approved in jira would be brought > to github as implementation specifications. > * Developers would work based on github issues and pull requests. > * Tracking and prioritising of the issues on github will be done using a > combination of the labels, milestones and projects. > > > Specifications: > > `Specifications' are a restructuring of the existing workflow that uses > jira epics and the documentation repository (not necessarily together or > in-sync). Specifications will be the single point of origin for any > enhancements (includes features and user stories) to be implemented in the > project codebases. A new repository has been > created to host the specifications in. This repository, like all others, > will be worked on via pull requests. Rohan, Nishanth and I, currently, will > be looking after the specifications (referred to as the `specification > maintainers' or `SMs' here onwards). This includes reviews for any > contributed specifications, conversion of mailing list > and jira discussions into specifications etc. > > Specifications would provide project-wide view of enhancements. This allows > for the impact on various sub-projects such as the framework, storage > system integrations, add-on stacks and components etc. to be scoped out. > >From specifications, issues would be created in the sub-project > repositories for implementation. > > > The Proposed Workflows: > > Feature proposals: > * Propose an enhancement via discussion on the mailing list, irc or > jira[1]. > * Send a pull request on the specifications repository with the details for > implementation. > * The SMs will ensure that the specifications map to a > corresponding feature or user story in jira. The entry would be created if > one doesn't exist. Discussions on jira would allow the specifications to be > prioritised for implementation. > > General development: > * Create (or help the SMs create) github issues on the > individual repositories based on the specifications. > * Report bugs as github issues directly. > * Send pull requests against the existing github issues with the issue > and/or the specification id clearly mentioned in the commit message. > > > [1] https://github.com/Tendrl/specifications > > -- > Mrugesh > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From mrugesh at brainfunked.org Tue Nov 29 09:26:12 2016 From: mrugesh at brainfunked.org (Mrugesh Karnik) Date: Tue, 29 Nov 2016 14:56:12 +0530 Subject: [Tendrl-devel] Tendrl Flow Definitions In-Reply-To: References: <885696456.190941.1479466632538.JavaMail.zimbra@redhat.com> Message-ID: Shubhendu, Anmol, please turn this into a specification. Impact on the framework and any other components can be scoped on the specification directly. On 23 November 2016 at 14:05, Rohan Kanade wrote: > > Tendrl aims to provide multi cluster (ceph, gluster etc) management via a > single API endpoint. We need to ensure the following things to enable that. > > - Node agents will always be cluster agnostic i.e. Node agents and their > definition file will be outside the cluster namespace and will remain the > same for that tendrl deployment, Hence we cannot aggregate the node agent > definition with the cluster definitions > > - Any updates to the node agent definitions in etcd by monitoring or any > other service can be discussed, these definitions under similar namespace > can be aggregated > . > > - The ceph and gluster integration have their own namespace which comes > with a cluster context, any updates to this namespace can be aggregated. > > > TL;DR: Multiple services (monitoring, node agent) can use one single path > to store their definitions in etcd if and only if they belong to the same > namespace. eg: tendrl.node_agent.objects and > tendrl.node_agent.monitoring.objects > > > Makes sense? > > On Fri, Nov 18, 2016 at 5:13 PM, Shubhendu Tripathi > wrote: > > > I agree and this is something which we discussed yesterday in detail. > > Request others to comment. > > > > Regards, > > Shubhendu > > > > > > > > On 11/18/2016 04:27 PM, Anmol Babu wrote: > > > > Hi all, > > > > I have a suggestion for a very small enhancement in the current flow framework. > > > > Current working procedure: > > > > 1. Any service/agent defining and implementing a flow pushes the corresponding flow definitions to its specific definitions etcd directory > > 2. The Agent/service flow framework reads its specific definition files and does all validations and executes the flow. > > > > ex: > > 1. Node-agent pushes tendrl_definitions_node_agent to /tendrl_definitions_node_agent in etcd, gluster integration pushes tendrl_definitions_gluster_integration to /tendrl_definitions_gluster_integration and so on... > > 2. node-agent is a service that reads flow definitions from /tendrl_definitions_node_agent in etcd and gluster-integration is a process that reads /tendrl_definitions_gluster_integration from etcd and currently each of them > > replicates the flow definition parsing logic(probably its already currently thought through to move generic sections of flow yaml parsing to tendrl/common) > > > > Drawback: > > > > Any component requiring to define its own flows will then need to implement(currently followed)/extend(from a generic one in tendrl/common -- probably already thought of) the flow framework > > and this requires the component to be run as a service as the flow framework in each of these component will only and always read only its specific definition and whenever there is a job(intended for it) > > in the etcd queue. > > > > Suggestion: > > > > 1. Push all flow definitions to a path under a common root directory say for ex: /tendrl_definitons/ in etcd > > 2. Now, when there's a job in the queue, the flow framework will make use of either: > > i. A part of the full qualified package name in the run parameter of the job > > or > > ii. A separate field to indicate the definitions namespace(i.e, the name under the /tendrl_definitions/ where the definition can be obtained from) > > to read definitions from appropriate etcd directory. > > 3. And from there the already existing flow framework as usual will extract and execute the pre, post and the normal flow atoms. > > > > Relevance of the suggestion: > > > > 1. There is a central application called tendrl/performance_monitoring which performs the task of > > i. Statistics aggregation > > ii. Exposing time series db data > > iii. Loading monitoring specific initial flow definitions to etcd > > 2. There will be a node-side library that defines atoms and flows specific to monitoring. This is carved out from tendrl/performance_monitoring application(separate rpms). > > The flow definition(yaml) being specific to monitoring, cannot be part of flow definitions(yaml) of any other project i.e, node_agent or even the gluster-integration or ceph-integration > > because monitoring is intended to be completely outside core stack. Now, given this, consider the case of a monitoring flow to generate collectd configurations on the nodes. In such a case > > the idea is to employ a python script(wrapper around jinja2 templating) that is executed as a command by the flow > > (This is not still generic as internally the command is not just a mere jinja2 wrapper but contains some specifics of collectd). > > So that the flow is very generic(with very specific parameters). But exposing > > this as a flow to execute any normal command can turn out to be a breach of security and hence the command needs to be hard-coded in the flow's respective atom which makes it still monitoring specific > > and hence needs to be part of an application that is specific to monitoring. But, with the current flow framework, just so that flow is executed, the flow implementer needs to be a service. > > But, with the small change suggested above the implementing application can be a mere stateless library(like tendrl/common) and any and every flow execution can be streamlined to be executed by the node-agent > > by just a simple python import of the specific installed python source which can be provided by the library(specific tendrl project). > > > > > > Please consider this idea and provide your valuable suggestions/feedback. > > > > > > Thanks, > > Anmol > > > > > > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From cblum at redhat.com Tue Nov 29 10:17:58 2016 From: cblum at redhat.com (Christopher Blum) Date: Tue, 29 Nov 2016 11:17:58 +0100 Subject: [Tendrl-devel] Tendrl UX designs for review In-Reply-To: References: Message-ID: <20FF5E7C-1115-462C-9F06-A61D3C2B3481@redhat.com> I know it has been a while, but adding my comments anyways ;) Regarding TEN-163: Rebalancing should be supported whenever we have a distributed sub-volume. That means, that we also rebalance once we extend a dispersed volume (erasure coding). This also applies when we extend a dispersed volume to a distributed-dispersed volume. Regarding TEN-164: * I heard that NFS-ganesha will be supported by RHS-C, this means we will have NFS v4 and we will need to at least activate IPv6 functionality (but don't necessarily need to configure NICs) * DNS resolvable Hosts is unfortunately very rare for the first installation - can we add an option to add the hostnames to all /etc/hosts files? (low prio) * IMO this should be 'Colocated journal' and 'Dedicated journal'. * Raw capacity in the "available Hosts" list, should only list available storage space for Ceph/Gluster usage (Not the OS disk) * If we can import a JSON file, we should be able to export the current settings somewhere as well. * Choosing the replica count in the 'Create Cluster' workflow is confusing. The user should/will be able to chose different replicas for different pools/volumes (RHCS/RHGS) later on. * Choosing a value for 'Optimized PG Count' is confusing because we don't know how many pools RHS-C will create. --> Better move this to the next window Regarding TEN-165: Should we be able to delete a volume with unreachable bricks? Who will clean up these bricks later on after we already decided to 'Delete data from all bricks'? Best Regards, Christopher Blum Storage Consultant Global Storage Practice, Red Hat +49 711 96 43 7009 > On 23 Nov 2016, at 22:19, Jeff Applewhite wrote: > > > Please have everyone take a look > > Thanks > > Jeff > > ---------- Forwarded message ---------- > From: Matt Carrano > > Date: Wednesday, November 23, 2016 > Subject: [Tendrl-devel] Tendrl UX designs for review > To: tendrl-devel at redhat.com > > > Three new UX designs are published for review. Please see the links below. > > Rebalance File Share (TEN-163): > https://redhat.invisionapp.com/share/AB94BNET6 > > Create Ceph Cluster (TEN-164): > https://redhat.invisionapp.com/share/2K8M4PQYZ > > Delete File Share (TEN-165): https://redhat.invisionapp.com/share/729GRP1W9 > > Comments may be left directly in the InVision documents. If you have never > commented in InVision before, you should read this article > https://support.invisionapp.com/hc/en-us/articles/209192426-How-do-I-comment-on-a-prototype- > > Please review and comment by COB on Wed Nov 30. We will be scheduling a > follow-up review meeting sometime in the next 2 weeks. > > Regards, > > Matt > > -- > Matt Carrano > Sr. Interaction Designer > Red Hat, Inc. > mcarrano at redhat.com > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > > > > -- > Jeff Applewhite > Principal Product Manager > > From vsarmila at redhat.com Tue Nov 29 11:52:16 2016 From: vsarmila at redhat.com (Sharmilla Abhilash) Date: Tue, 29 Nov 2016 17:22:16 +0530 Subject: [Tendrl-devel] Calamari and the Tendrl Ceph Bridge In-Reply-To: References: Message-ID: On Fri, Oct 28, 2016 at 11:06 PM, Gregory Meno wrote: > Looks good to me. > > On Thu, Oct 20, 2016 at 5:27 AM, Rohan Kanade wrote: > > These are the minimum requirements Tendrl has from calamari. If we are > > agreeing on these, we can file detailed BZ's. > Rohan, all the required BZ's has been filed here ? Gregory, do you you have a time-line when the agreed upon requirements will be available ? thanks! From jspray at redhat.com Tue Nov 29 14:04:07 2016 From: jspray at redhat.com (John Spray) Date: Tue, 29 Nov 2016 14:04:07 +0000 Subject: [Tendrl-devel] Tendrl UX designs for review In-Reply-To: <20FF5E7C-1115-462C-9F06-A61D3C2B3481@redhat.com> References: <20FF5E7C-1115-462C-9F06-A61D3C2B3481@redhat.com> Message-ID: On Tue, Nov 29, 2016 at 10:17 AM, Christopher Blum wrote: > I know it has been a while, but adding my comments anyways ;) > > Regarding TEN-163: > Rebalancing should be supported whenever we have a distributed sub-volume. That means, that we also rebalance once we extend a dispersed volume (erasure coding). This also applies when we extend a dispersed volume to a distributed-dispersed volume. > > Regarding TEN-164: > * I heard that NFS-ganesha will be supported by RHS-C, this means we will have NFS v4 and we will need to at least activate IPv6 functionality (but don't necessarily need to configure NICs) > * DNS resolvable Hosts is unfortunately very rare for the first installation - can we add an option to add the hostnames to all /etc/hosts files? (low prio) > > * IMO this should be 'Colocated journal' and 'Dedicated journal'. > * Raw capacity in the "available Hosts" list, should only list available storage space for Ceph/Gluster usage (Not the OS disk) > * If we can import a JSON file, we should be able to export the current settings somewhere as well. > * Choosing the replica count in the 'Create Cluster' workflow is confusing. The user should/will be able to chose different replicas for different pools/volumes (RHCS/RHGS) later on. > * Choosing a value for 'Optimized PG Count' is confusing because we don't know how many pools RHS-C will create. > --> Better move this to the next window I agree with all these Ceph points (and I added lots of other comments to the wireframe). John > > Regarding TEN-165: > Should we be able to delete a volume with unreachable bricks? Who will clean up these bricks later on after we already decided to 'Delete data from all bricks'? > > Best Regards, > Christopher Blum > Storage Consultant > Global Storage Practice, Red Hat > > +49 711 96 43 7009 > >> On 23 Nov 2016, at 22:19, Jeff Applewhite wrote: >> >> >> Please have everyone take a look >> >> Thanks >> >> Jeff >> >> ---------- Forwarded message ---------- >> From: Matt Carrano > >> Date: Wednesday, November 23, 2016 >> Subject: [Tendrl-devel] Tendrl UX designs for review >> To: tendrl-devel at redhat.com >> >> >> Three new UX designs are published for review. Please see the links below. >> >> Rebalance File Share (TEN-163): >> https://redhat.invisionapp.com/share/AB94BNET6 >> >> Create Ceph Cluster (TEN-164): >> https://redhat.invisionapp.com/share/2K8M4PQYZ >> >> Delete File Share (TEN-165): https://redhat.invisionapp.com/share/729GRP1W9 >> >> Comments may be left directly in the InVision documents. If you have never >> commented in InVision before, you should read this article >> https://support.invisionapp.com/hc/en-us/articles/209192426-How-do-I-comment-on-a-prototype- >> >> Please review and comment by COB on Wed Nov 30. We will be scheduling a >> follow-up review meeting sometime in the next 2 weeks. >> >> Regards, >> >> Matt >> >> -- >> Matt Carrano >> Sr. Interaction Designer >> Red Hat, Inc. >> mcarrano at redhat.com >> _______________________________________________ >> Tendrl-devel mailing list >> Tendrl-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/tendrl-devel >> >> >> >> -- >> Jeff Applewhite >> Principal Product Manager >> >> > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From shtripat at redhat.com Wed Nov 30 04:05:32 2016 From: shtripat at redhat.com (Shubhendu Tripathi) Date: Wed, 30 Nov 2016 09:35:32 +0530 Subject: [Tendrl-devel] Regarding some additional data points in the state output In-Reply-To: References: <5e1609e4-cfc3-4dc3-f421-ac51f56b9d64@redhat.com> <1988c0ba-6dfb-cdb3-a70e-4e51f02830e0@redhat.com> Message-ID: Dear Rohan, I accept that the below format is a valid ini file format syntactically, but semantically its screwing up volumes listing. The "Volumes" list returns [Volumes] Volume1.name: test-vol Volume1.id: 7942d008-e300-4fd9-8af0-5a118afd8d3d Volume1.type: Distribute Volume1.transport_type: tcp Volume1.status: Started Volume1.brickcount: 1 Volume1.Brick1.path: 172.17.0.2:/tmp/b1 Volume1.Brick1.hostname: 172.17.0.2 Volume1.Brick1.port: 49153 Volume1.Brick1.rdma_port: 0 Volume1.Brick1.status: Started Volume1.Brick1.signedin: True Volume1.snap_count: 0 Volume1.stripe_count: 1 Volume1.replica_count: 1 Volume1.subvol_count: 1 Volume1.arbiter_count: 0 Volume1.disperse_count: 0 Volume1.redundancy_count: 0 Volume1.quorum_status: not_applicable Volume1.snapd_svc.online_status: Offline Volume1.snapd_svc.inited: True Volume1.rebalance.id: 00000000-0000-0000-0000-000000000000 Volume1.rebalance.status: not_started Volume1.rebalance.failures: 0 Volume1.rebalance.skipped: 0 Volume1.rebalance.lookedup: 0 Volume1.rebalance.files: 0 Volume1.rebalance.data: 0Bytes Also the "Volume1.options" returns the volume2 details mingled within as below features.barrier: on transport.address-family: inet performance.readdir-ahead: on nfs.disable: on Volume2.name: test-vol1 Volume2.id: 35854708-bb72-45a5-bdbd-77c51e5ebfb9 Volume2.type: Distribute Volume2.transport_type: tcp Volume2.status: Started Volume2.brickcount: 1 Volume2.Brick1.path: 172.17.0.2:/tmp/b2 Volume2.Brick1.hostname: 172.17.0.2 Volume2.Brick1.port: 49152 Volume2.Brick1.rdma_port: 0 Volume2.Brick1.status: Started Volume2.Brick1.signedin: True Volume2.snap_count: 0 Volume2.stripe_count: 1 Volume2.replica_count: 1 Volume2.subvol_count: 1 Volume2.arbiter_count: 0 Volume2.disperse_count: 0 Volume2.redundancy_count: 0 Volume2.quorum_status: not_applicable Volume2.snapd_svc.online_status: Offline Volume2.snapd_svc.inited: True Volume2.rebalance.id: 00000000-0000-0000-0000-000000000000 Volume2.rebalance.status: not_started Volume2.rebalance.failures: 0 Volume2.rebalance.skipped: 0 Volume2.rebalance.lookedup: 0 Volume2.rebalance.files: 0 Volume2.rebalance.data: 0Bytes Tried debugging a little the parser for this ini file and it looks like sector/sections are formed based on [] brackets and anything below one section (till next [] found) is treated as one section. Instead, the flatted structure like "Volume1.options.nfs.disable: on" would have been an easier option to parse and code change tendrl side. At the moment I dont find a way to resolve this mingled sections and handling within tendrl parser. I have tried some tweaking in parser but looks like sections are formed underneath using the library for ini parser. Comments?? Regards, Shubhendu On 11/22/2016 08:16 PM, Rohan Kanade wrote: > Sample: > START>>> > > [Global] > MYUUID: 6bbf8ac2-22a0-4f08-b986-fe75aea9f654 > op-version: 40000 > > [Global options] > > [Peers] > > [Volumes] > Volume1.name: test-vol > Volume1.id: 7942d008-e300-4fd9-8af0-5a118afd8d3d > Volume1.type: Distribute > Volume1.transport_type: tcp > Volume1.status: Started > Volume1.brickcount: 1 > Volume1.Brick1.path: 172.17.0.2:/tmp/b1 > Volume1.Brick1.hostname: 172.17.0.2 > Volume1.Brick1.port: 49153 > Volume1.Brick1.rdma_port: 0 > Volume1.Brick1.status: Started > Volume1.Brick1.signedin: True > Volume1.snap_count: 0 > Volume1.stripe_count: 1 > Volume1.replica_count: 1 > Volume1.subvol_count: 1 > Volume1.arbiter_count: 0 > Volume1.disperse_count: 0 > Volume1.redundancy_count: 0 > Volume1.quorum_status: not_applicable > Volume1.snapd_svc.online_status: Offline > Volume1.snapd_svc.inited: True > Volume1.rebalance.id: 00000000-0000-0000-0000-000000000000 > Volume1.rebalance.status: not_started > Volume1.rebalance.failures: 0 > Volume1.rebalance.skipped: 0 > Volume1.rebalance.lookedup: 0 > Volume1.rebalance.files: 0 > Volume1.rebalance.data: 0Bytes > [Volume1.options] > features.barrier: on > transport.address-family: inet > performance.readdir-ahead: on > nfs.disable: on > > Volume2.name: test-vol1 > Volume2.id: 35854708-bb72-45a5-bdbd-77c51e5ebfb9 > Volume2.type: Distribute > Volume2.transport_type: tcp > Volume2.status: Started > Volume2.brickcount: 1 > Volume2.Brick1.path: 172.17.0.2:/tmp/b2 > Volume2.Brick1.hostname: 172.17.0.2 > Volume2.Brick1.port: 49152 > Volume2.Brick1.rdma_port: 0 > Volume2.Brick1.status: Started > Volume2.Brick1.signedin: True > Volume2.snap_count: 0 > Volume2.stripe_count: 1 > Volume2.replica_count: 1 > Volume2.subvol_count: 1 > Volume2.arbiter_count: 0 > Volume2.disperse_count: 0 > Volume2.redundancy_count: 0 > Volume2.quorum_status: not_applicable > Volume2.snapd_svc.online_status: Offline > Volume2.snapd_svc.inited: True > Volume2.rebalance.id: 00000000-0000-0000-0000-000000000000 > Volume2.rebalance.status: not_started > Volume2.rebalance.failures: 0 > Volume2.rebalance.skipped: 0 > Volume2.rebalance.lookedup: 0 > Volume2.rebalance.files: 0 > Volume2.rebalance.data: 0Bytes > [Volume2.options] > transport.address-family: inet > performance.readdir-ahead: on > nfs.disable: on > > > [Services] > svc1.name: glustershd > svc1.online_status: Offline > > svc2.name: nfs > svc2.online_status: Offline > > svc3.name: bitd > svc3.online_status: Offline > > svc4.name: scrub > svc4.online_status: Offline > > svc5.name: quotad > svc5.online_status: Offline > > > [Misc] > Base port: 49152 > Last allocated port: 49153 > > < > On Tue, Nov 22, 2016 at 2:06 PM, Rohan Kanade wrote: >> Also, please provide a full state dump example with this patch included, > easier for Tendrl devs to get started without deploying this patch >> On Tue, Nov 22, 2016 at 1:44 PM, Rohan Kanade wrote: >>> Id prefer the first option >>> >>> >>> [Volumes] >>> Volume1.name: tv1 >>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>> Volume1.type: Distribute >>> >>> Volume1.rebalance.files: 0 >>> Volume1.rebalance.data: 0Bytes >>> [Volume1.options] >>> nfs.disable: on >>> performance.readdir-ahead: on >>> transport.address-family: inet >>> features.uss: on >>> >>> Volume2.name: tv2 >>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>> Volume2.type: Distribute >>> ......... >>> ......... >>> >>> >>> This would require minor changes to tendrl/gluster-integration > definition files and code. I will draw up a spec on Tendrl/specifications > to document the changes required. Please go ahead with your patch >>> Thanks >>> >>> On Tue, Nov 22, 2016 at 9:18 AM, Atin Mukherjee > wrote: >>>> We are awaiting final confirmation from Rohan. Samikshan has kept the >>>> changes ready and will push it to gerrit once we hear from Rohan. >>>> >>>> On Mon, Nov 21, 2016 at 12:54 PM, Shubhendu Tripathi < > shtripat at redhat.com> >>>> wrote: >>>> >>>>> Looking at options, I feel option-2 would be more feasible and might > not >>>>> need code changes in tendrl. >>>>> But still lets wait for the confirmation from Rohan. >>>>> >>>>> Regards, >>>>> Shubhendu >>>>> >>>>> >>>>> >>>>> On 11/21/2016 12:43 PM, Atin Mukherjee wrote: >>>>> >>>>>> +tendrl-devel >>>>>> >>>>>> On Mon, Nov 21, 2016 at 12:41 PM, Samikshan Bairagya < > sbairagy at redhat.com >>>>>> wrote: >>>>>> >>>>>> Hey Rohan, >>>>>>> So the current get-state CLI misses volume specific options in its >>>>>>> output. >>>>>>> Somehow I missed it while coming up with the implementation. This > patch >>>>>>> by >>>>>>> Atin is a fix for that: http://review.gluster.org/#/c/15889/1. The >>>>>>> following example shows how this patch would add these new data > points >>>>>>> and >>>>>>> how that would change the existing format: >>>>>>> >>>>>>> [Volumes] >>>>>>> Volume1.name: tv1 >>>>>>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>>>>>> Volume1.type: Distribute >>>>>>> >>>>>>> Volume1.rebalance.files: 0 >>>>>>> Volume1.rebalance.data: 0Bytes >>>>>>> [Volume1.options] >>>>>>> nfs.disable: on >>>>>>> performance.readdir-ahead: on >>>>>>> transport.address-family: inet >>>>>>> features.uss: on >>>>>>> >>>>>>> Volume2.name: tv2 >>>>>>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>>>>>> Volume2.type: Distribute >>>>>>> ......... >>>>>>> ......... >>>>>>> >>>>>>> So essentially there would be a new section for every volume that > would >>>>>>> list the option names and corresponding values. Would adding this > change >>>>>>> still keep the get-state output parseable from Tendrl POV? >>>>>>> >>>>>>> Or would an output like the following make more sense? Let us know. >>>>>>> Thanks. >>>>>>> >>>>>>> [Volumes] >>>>>>> Volume1.name: tv1 >>>>>>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>>>>>> Volume1.type: Distribute >>>>>>> >>>>>>> Volume1.rebalance.files: 0 >>>>>>> Volume1.rebalance.data: 0Bytes >>>>>>> Volume1.options.nfs.disable: on >>>>>>> Volume1.options.performance.readdir-ahead: on >>>>>>> Volume1.options.transport.address-family: inet >>>>>>> Volume1.options.features.uss: on >>>>>>> >>>>>>> Volume2.name: tv2 >>>>>>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>>>>>> Volume2.type: Distribute >>>>>>> ......... >>>>>>> ......... >>>>>>> >>>>>>> >>>>>>> ~ Samikshan >>>>>>> >>>>>>> >>>>>> >>>>> _______________________________________________ >>>>> Tendrl-devel mailing list >>>>> Tendrl-devel at redhat.com >>>>> https://www.redhat.com/mailman/listinfo/tendrl-devel >>>>> >>>> >>>> >>>> -- >>>> >>>> ~ Atin (atinm) >>>> _______________________________________________ >>>> Tendrl-devel mailing list >>>> Tendrl-devel at redhat.com >>>> https://www.redhat.com/mailman/listinfo/tendrl-devel >>> > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From sankarshan.mukhopadhyay at gmail.com Wed Nov 30 04:19:13 2016 From: sankarshan.mukhopadhyay at gmail.com (Sankarshan Mukhopadhyay) Date: Wed, 30 Nov 2016 09:49:13 +0530 Subject: [Tendrl-devel] Regarding some additional data points in the state output In-Reply-To: References: <5e1609e4-cfc3-4dc3-f421-ac51f56b9d64@redhat.com> <1988c0ba-6dfb-cdb3-a70e-4e51f02830e0@redhat.com> Message-ID: On Wed, Nov 30, 2016 at 9:35 AM, Shubhendu Tripathi wrote: [snip] > Tried debugging a little the parser for this ini file and it looks like > sector/sections are formed based on [] brackets and anything below one > section (till next [] found) is treated as one section. > > Instead, the flatted structure like "Volume1.options.nfs.disable: on" would > have been an easier option to parse and code change tendrl side. If there's a change in the offing, it is probably best to quickly work with the Gluster developers. The patch which enabled the structure was discussed with the Tendrl team before being merged. -- sankarshan mukhopadhyay From rkanade at redhat.com Wed Nov 30 07:13:23 2016 From: rkanade at redhat.com (Rohan Kanade) Date: Wed, 30 Nov 2016 12:43:23 +0530 Subject: [Tendrl-devel] Regarding some additional data points in the state output In-Reply-To: References: <5e1609e4-cfc3-4dc3-f421-ac51f56b9d64@redhat.com> <1988c0ba-6dfb-cdb3-a70e-4e51f02830e0@redhat.com> Message-ID: 2 ways to tackle this: 1) Tweak our ini parser such that, when it sees section like "[Volume.options]", take the "Volume.options" text and prefix it to the options given below until we find another "Volume.*" key/value. This will basically transform the output file to what it looked like as the option 1 presented by gluster folks to us 2) If this can be patched and output can be changed to look like below, we are good with that too. [Volumes] Volume1.name: tv1 Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 Volume1.type: Distribute Volume1.rebalance.files: 0 Volume1.rebalance.data: 0Bytes Volume1.options.nfs.disable: on Volume1.options.performance.readdir-ahead: on Volume1.options.transport.address-family: inet Volume1.options.features.uss: on Volume2.name: tv2 Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037aVolume2.type: Distribute On Wed, Nov 30, 2016 at 9:35 AM, Shubhendu Tripathi wrote: > Dear Rohan, > > I accept that the below format is a valid ini file format syntactically, > but semantically its screwing up volumes listing. > The "Volumes" list returns > > [Volumes] > Volume1.name: test-vol > Volume1.id: 7942d008-e300-4fd9-8af0-5a118afd8d3d > Volume1.type: Distribute > Volume1.transport_type: tcp > Volume1.status: Started > Volume1.brickcount: 1 > Volume1.Brick1.path: 172.17.0.2:/tmp/b1 > Volume1.Brick1.hostname: 172.17.0.2 > Volume1.Brick1.port: 49153 > Volume1.Brick1.rdma_port: 0 > Volume1.Brick1.status: Started > Volume1.Brick1.signedin: True > Volume1.snap_count: 0 > Volume1.stripe_count: 1 > Volume1.replica_count: 1 > Volume1.subvol_count: 1 > Volume1.arbiter_count: 0 > Volume1.disperse_count: 0 > Volume1.redundancy_count: 0 > Volume1.quorum_status: not_applicable > Volume1.snapd_svc.online_status: Offline > Volume1.snapd_svc.inited: TrueVolume1.rebalance.id: 00000000-0000-0000-0000-000000000000 > Volume1.rebalance.status: not_started > Volume1.rebalance.failures: 0 > Volume1.rebalance.skipped: 0 > Volume1.rebalance.lookedup: 0 > Volume1.rebalance.files: 0 > Volume1.rebalance.data: 0Bytes > > Also the "Volume1.options" returns the volume2 details mingled within as > below > > > features.barrier: on > transport.address-family: inet > performance.readdir-ahead: on > nfs.disable: on > > Volume2.name: test-vol1 > Volume2.id: 35854708-bb72-45a5-bdbd-77c51e5ebfb9 > Volume2.type: Distribute > Volume2.transport_type: tcp > Volume2.status: Started > Volume2.brickcount: 1 > Volume2.Brick1.path: 172.17.0.2:/tmp/b2 > Volume2.Brick1.hostname: 172.17.0.2 > Volume2.Brick1.port: 49152 > Volume2.Brick1.rdma_port: 0 > Volume2.Brick1.status: Started > Volume2.Brick1.signedin: True > Volume2.snap_count: 0 > Volume2.stripe_count: 1 > Volume2.replica_count: 1 > Volume2.subvol_count: 1 > Volume2.arbiter_count: 0 > Volume2.disperse_count: 0 > Volume2.redundancy_count: 0 > Volume2.quorum_status: not_applicable > Volume2.snapd_svc.online_status: Offline > Volume2.snapd_svc.inited: TrueVolume2.rebalance.id: 00000000-0000-0000-0000-000000000000 > Volume2.rebalance.status: not_started > Volume2.rebalance.failures: 0 > Volume2.rebalance.skipped: 0 > Volume2.rebalance.lookedup: 0 > Volume2.rebalance.files: 0 > Volume2.rebalance.data: 0Bytes > > Tried debugging a little the parser for this ini file and it looks like > sector/sections are formed based on [] brackets and anything below one > section (till next [] found) is treated as one section. > > Instead, the flatted structure like "Volume1.options.nfs.disable: on" > would have been an easier option to parse and code change tendrl side. > > At the moment I dont find a way to resolve this mingled sections and > handling within tendrl parser. I have tried some tweaking in parser but > looks like sections are formed underneath using the library for ini parser. > > Comments?? > > Regards, > Shubhendu > > > > On 11/22/2016 08:16 PM, Rohan Kanade wrote: > > Sample: > START>>> > > [Global] > MYUUID: 6bbf8ac2-22a0-4f08-b986-fe75aea9f654 > op-version: 40000 > > [Global options] > > [Peers] > > [Volumes] > Volume1.name: test-vol > Volume1.id: 7942d008-e300-4fd9-8af0-5a118afd8d3d > Volume1.type: Distribute > Volume1.transport_type: tcp > Volume1.status: Started > Volume1.brickcount: 1 > Volume1.Brick1.path: 172.17.0.2:/tmp/b1 > Volume1.Brick1.hostname: 172.17.0.2 > Volume1.Brick1.port: 49153 > Volume1.Brick1.rdma_port: 0 > Volume1.Brick1.status: Started > Volume1.Brick1.signedin: True > Volume1.snap_count: 0 > Volume1.stripe_count: 1 > Volume1.replica_count: 1 > Volume1.subvol_count: 1 > Volume1.arbiter_count: 0 > Volume1.disperse_count: 0 > Volume1.redundancy_count: 0 > Volume1.quorum_status: not_applicable > Volume1.snapd_svc.online_status: Offline > Volume1.snapd_svc.inited: TrueVolume1.rebalance.id: 00000000-0000-0000-0000-000000000000 > Volume1.rebalance.status: not_started > Volume1.rebalance.failures: 0 > Volume1.rebalance.skipped: 0 > Volume1.rebalance.lookedup: 0 > Volume1.rebalance.files: 0 > Volume1.rebalance.data: 0Bytes > [Volume1.options] > features.barrier: on > transport.address-family: inet > performance.readdir-ahead: on > nfs.disable: on > > Volume2.name: test-vol1 > Volume2.id: 35854708-bb72-45a5-bdbd-77c51e5ebfb9 > Volume2.type: Distribute > Volume2.transport_type: tcp > Volume2.status: Started > Volume2.brickcount: 1 > Volume2.Brick1.path: 172.17.0.2:/tmp/b2 > Volume2.Brick1.hostname: 172.17.0.2 > Volume2.Brick1.port: 49152 > Volume2.Brick1.rdma_port: 0 > Volume2.Brick1.status: Started > Volume2.Brick1.signedin: True > Volume2.snap_count: 0 > Volume2.stripe_count: 1 > Volume2.replica_count: 1 > Volume2.subvol_count: 1 > Volume2.arbiter_count: 0 > Volume2.disperse_count: 0 > Volume2.redundancy_count: 0 > Volume2.quorum_status: not_applicable > Volume2.snapd_svc.online_status: Offline > Volume2.snapd_svc.inited: TrueVolume2.rebalance.id: 00000000-0000-0000-0000-000000000000 > Volume2.rebalance.status: not_started > Volume2.rebalance.failures: 0 > Volume2.rebalance.skipped: 0 > Volume2.rebalance.lookedup: 0 > Volume2.rebalance.files: 0 > Volume2.rebalance.data: 0Bytes > [Volume2.options] > transport.address-family: inet > performance.readdir-ahead: on > nfs.disable: on > > > [Services]svc1.name: glustershd > svc1.online_status: Offline > svc2.name: nfs > svc2.online_status: Offline > svc3.name: bitd > svc3.online_status: Offline > svc4.name: scrub > svc4.online_status: Offline > svc5.name: quotad > svc5.online_status: Offline > > > [Misc] > Base port: 49152 > Last allocated port: 49153 > > < > On Tue, Nov 22, 2016 at 2:06 PM, Rohan Kanade wrote: > > Also, please provide a full state dump example with this patch included, > > easier for Tendrl devs to get started without deploying this patch > > On Tue, Nov 22, 2016 at 1:44 PM, Rohan Kanade wrote: > > Id prefer the first option > > > [Volumes] > Volume1.name: tv1 > Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 > Volume1.type: Distribute > > Volume1.rebalance.files: 0 > Volume1.rebalance.data: 0Bytes > [Volume1.options] > nfs.disable: on > performance.readdir-ahead: on > transport.address-family: inet > features.uss: on > > Volume2.name: tv2 > Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a > Volume2.type: Distribute > ......... > ......... > > > This would require minor changes to tendrl/gluster-integration > > definition files and code. I will draw up a spec on Tendrl/specifications > to document the changes required. Please go ahead with your patch > > Thanks > > On Tue, Nov 22, 2016 at 9:18 AM, Atin Mukherjee > > wrote: > > We are awaiting final confirmation from Rohan. Samikshan has kept the > changes ready and will push it to gerrit once we hear from Rohan. > > On Mon, Nov 21, 2016 at 12:54 PM, Shubhendu Tripathi < > > shtripat at redhat.com> > > wrote: > > > Looking at options, I feel option-2 would be more feasible and might > > not > > need code changes in tendrl. > But still lets wait for the confirmation from Rohan. > > Regards, > Shubhendu > > > > On 11/21/2016 12:43 PM, Atin Mukherjee wrote: > > > +tendrl-devel > > On Mon, Nov 21, 2016 at 12:41 PM, Samikshan Bairagya < > > sbairagy at redhat.com > > wrote: > > Hey Rohan, > > So the current get-state CLI misses volume specific options in its > output. > Somehow I missed it while coming up with the implementation. This > > patch > > by > Atin is a fix for that: http://review.gluster.org/#/c/15889/1. The > following example shows how this patch would add these new data > > points > > and > how that would change the existing format: > > [Volumes] > Volume1.name: tv1 > Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 > Volume1.type: Distribute > > Volume1.rebalance.files: 0 > Volume1.rebalance.data: 0Bytes > [Volume1.options] > nfs.disable: on > performance.readdir-ahead: on > transport.address-family: inet > features.uss: on > > Volume2.name: tv2 > Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a > Volume2.type: Distribute > ......... > ......... > > So essentially there would be a new section for every volume that > > would > > list the option names and corresponding values. Would adding this > > change > > still keep the get-state output parseable from Tendrl POV? > > Or would an output like the following make more sense? Let us know. > Thanks. > > [Volumes] > Volume1.name: tv1 > Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 > Volume1.type: Distribute > > Volume1.rebalance.files: 0 > Volume1.rebalance.data: 0Bytes > Volume1.options.nfs.disable: on > Volume1.options.performance.readdir-ahead: on > Volume1.options.transport.address-family: inet > Volume1.options.features.uss: on > > Volume2.name: tv2 > Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a > Volume2.type: Distribute > ......... > ......... > > > ~ Samikshan > > > > _______________________________________________ > Tendrl-devel mailing listTendrl-devel at redhat.comhttps://www.redhat.com/mailman/listinfo/tendrl-devel > > -- > > ~ Atin (atinm) > _______________________________________________ > Tendrl-devel mailing listTendrl-devel at redhat.comhttps://www.redhat.com/mailman/listinfo/tendrl-devel > > _______________________________________________ > Tendrl-devel mailing listTendrl-devel at redhat.comhttps://www.redhat.com/mailman/listinfo/tendrl-devel > > > From shtripat at redhat.com Wed Nov 30 07:31:15 2016 From: shtripat at redhat.com (Shubhendu Tripathi) Date: Wed, 30 Nov 2016 13:01:15 +0530 Subject: [Tendrl-devel] Regarding some additional data points in the state output In-Reply-To: References: <5e1609e4-cfc3-4dc3-f421-ac51f56b9d64@redhat.com> <1988c0ba-6dfb-cdb3-a70e-4e51f02830e0@redhat.com> Message-ID: The ini parser is a generic utility in tendrl and I prefer not to tweak it only for this reason. I would rather prefer option two mentioned below. Regards, Shubhendu On 11/30/2016 12:43 PM, Rohan Kanade wrote: > 2 ways to tackle this: > > 1) Tweak our ini parser such that, when it sees section like > "[Volume.options]", take the "Volume.options" text and > prefix it to the options given below until we find another > "Volume.*" key/value. > > This will basically transform the output file to what it looked like as the > option 1 presented by gluster folks to us > > 2) If this can be patched and output can be changed to look like below, we > are good with that too. > > [Volumes] > Volume1.name: tv1 > Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 > Volume1.type: Distribute > > Volume1.rebalance.files: 0 > Volume1.rebalance.data: 0Bytes > Volume1.options.nfs.disable: on > Volume1.options.performance.readdir-ahead: on > Volume1.options.transport.address-family: inet > Volume1.options.features.uss: on > Volume2.name: tv2 > Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037aVolume2.type: Distribute > > On Wed, Nov 30, 2016 at 9:35 AM, Shubhendu Tripathi > wrote: > >> Dear Rohan, >> >> I accept that the below format is a valid ini file format syntactically, >> but semantically its screwing up volumes listing. >> The "Volumes" list returns >> >> [Volumes] >> Volume1.name: test-vol >> Volume1.id: 7942d008-e300-4fd9-8af0-5a118afd8d3d >> Volume1.type: Distribute >> Volume1.transport_type: tcp >> Volume1.status: Started >> Volume1.brickcount: 1 >> Volume1.Brick1.path: 172.17.0.2:/tmp/b1 >> Volume1.Brick1.hostname: 172.17.0.2 >> Volume1.Brick1.port: 49153 >> Volume1.Brick1.rdma_port: 0 >> Volume1.Brick1.status: Started >> Volume1.Brick1.signedin: True >> Volume1.snap_count: 0 >> Volume1.stripe_count: 1 >> Volume1.replica_count: 1 >> Volume1.subvol_count: 1 >> Volume1.arbiter_count: 0 >> Volume1.disperse_count: 0 >> Volume1.redundancy_count: 0 >> Volume1.quorum_status: not_applicable >> Volume1.snapd_svc.online_status: Offline >> Volume1.snapd_svc.inited: TrueVolume1.rebalance.id: 00000000-0000-0000-0000-000000000000 >> Volume1.rebalance.status: not_started >> Volume1.rebalance.failures: 0 >> Volume1.rebalance.skipped: 0 >> Volume1.rebalance.lookedup: 0 >> Volume1.rebalance.files: 0 >> Volume1.rebalance.data: 0Bytes >> >> Also the "Volume1.options" returns the volume2 details mingled within as >> below >> >> >> features.barrier: on >> transport.address-family: inet >> performance.readdir-ahead: on >> nfs.disable: on >> >> Volume2.name: test-vol1 >> Volume2.id: 35854708-bb72-45a5-bdbd-77c51e5ebfb9 >> Volume2.type: Distribute >> Volume2.transport_type: tcp >> Volume2.status: Started >> Volume2.brickcount: 1 >> Volume2.Brick1.path: 172.17.0.2:/tmp/b2 >> Volume2.Brick1.hostname: 172.17.0.2 >> Volume2.Brick1.port: 49152 >> Volume2.Brick1.rdma_port: 0 >> Volume2.Brick1.status: Started >> Volume2.Brick1.signedin: True >> Volume2.snap_count: 0 >> Volume2.stripe_count: 1 >> Volume2.replica_count: 1 >> Volume2.subvol_count: 1 >> Volume2.arbiter_count: 0 >> Volume2.disperse_count: 0 >> Volume2.redundancy_count: 0 >> Volume2.quorum_status: not_applicable >> Volume2.snapd_svc.online_status: Offline >> Volume2.snapd_svc.inited: TrueVolume2.rebalance.id: 00000000-0000-0000-0000-000000000000 >> Volume2.rebalance.status: not_started >> Volume2.rebalance.failures: 0 >> Volume2.rebalance.skipped: 0 >> Volume2.rebalance.lookedup: 0 >> Volume2.rebalance.files: 0 >> Volume2.rebalance.data: 0Bytes >> >> Tried debugging a little the parser for this ini file and it looks like >> sector/sections are formed based on [] brackets and anything below one >> section (till next [] found) is treated as one section. >> >> Instead, the flatted structure like "Volume1.options.nfs.disable: on" >> would have been an easier option to parse and code change tendrl side. >> >> At the moment I dont find a way to resolve this mingled sections and >> handling within tendrl parser. I have tried some tweaking in parser but >> looks like sections are formed underneath using the library for ini parser. >> >> Comments?? >> >> Regards, >> Shubhendu >> >> >> >> On 11/22/2016 08:16 PM, Rohan Kanade wrote: >> >> Sample: >> START>>> >> >> [Global] >> MYUUID: 6bbf8ac2-22a0-4f08-b986-fe75aea9f654 >> op-version: 40000 >> >> [Global options] >> >> [Peers] >> >> [Volumes] >> Volume1.name: test-vol >> Volume1.id: 7942d008-e300-4fd9-8af0-5a118afd8d3d >> Volume1.type: Distribute >> Volume1.transport_type: tcp >> Volume1.status: Started >> Volume1.brickcount: 1 >> Volume1.Brick1.path: 172.17.0.2:/tmp/b1 >> Volume1.Brick1.hostname: 172.17.0.2 >> Volume1.Brick1.port: 49153 >> Volume1.Brick1.rdma_port: 0 >> Volume1.Brick1.status: Started >> Volume1.Brick1.signedin: True >> Volume1.snap_count: 0 >> Volume1.stripe_count: 1 >> Volume1.replica_count: 1 >> Volume1.subvol_count: 1 >> Volume1.arbiter_count: 0 >> Volume1.disperse_count: 0 >> Volume1.redundancy_count: 0 >> Volume1.quorum_status: not_applicable >> Volume1.snapd_svc.online_status: Offline >> Volume1.snapd_svc.inited: TrueVolume1.rebalance.id: 00000000-0000-0000-0000-000000000000 >> Volume1.rebalance.status: not_started >> Volume1.rebalance.failures: 0 >> Volume1.rebalance.skipped: 0 >> Volume1.rebalance.lookedup: 0 >> Volume1.rebalance.files: 0 >> Volume1.rebalance.data: 0Bytes >> [Volume1.options] >> features.barrier: on >> transport.address-family: inet >> performance.readdir-ahead: on >> nfs.disable: on >> >> Volume2.name: test-vol1 >> Volume2.id: 35854708-bb72-45a5-bdbd-77c51e5ebfb9 >> Volume2.type: Distribute >> Volume2.transport_type: tcp >> Volume2.status: Started >> Volume2.brickcount: 1 >> Volume2.Brick1.path: 172.17.0.2:/tmp/b2 >> Volume2.Brick1.hostname: 172.17.0.2 >> Volume2.Brick1.port: 49152 >> Volume2.Brick1.rdma_port: 0 >> Volume2.Brick1.status: Started >> Volume2.Brick1.signedin: True >> Volume2.snap_count: 0 >> Volume2.stripe_count: 1 >> Volume2.replica_count: 1 >> Volume2.subvol_count: 1 >> Volume2.arbiter_count: 0 >> Volume2.disperse_count: 0 >> Volume2.redundancy_count: 0 >> Volume2.quorum_status: not_applicable >> Volume2.snapd_svc.online_status: Offline >> Volume2.snapd_svc.inited: TrueVolume2.rebalance.id: 00000000-0000-0000-0000-000000000000 >> Volume2.rebalance.status: not_started >> Volume2.rebalance.failures: 0 >> Volume2.rebalance.skipped: 0 >> Volume2.rebalance.lookedup: 0 >> Volume2.rebalance.files: 0 >> Volume2.rebalance.data: 0Bytes >> [Volume2.options] >> transport.address-family: inet >> performance.readdir-ahead: on >> nfs.disable: on >> >> >> [Services]svc1.name: glustershd >> svc1.online_status: Offline >> svc2.name: nfs >> svc2.online_status: Offline >> svc3.name: bitd >> svc3.online_status: Offline >> svc4.name: scrub >> svc4.online_status: Offline >> svc5.name: quotad >> svc5.online_status: Offline >> >> >> [Misc] >> Base port: 49152 >> Last allocated port: 49153 >> >> <> >> On Tue, Nov 22, 2016 at 2:06 PM, Rohan Kanade wrote: >> >> Also, please provide a full state dump example with this patch included, >> >> easier for Tendrl devs to get started without deploying this patch >> >> On Tue, Nov 22, 2016 at 1:44 PM, Rohan Kanade wrote: >> >> Id prefer the first option >> >> >> [Volumes] >> Volume1.name: tv1 >> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >> Volume1.type: Distribute >> >> Volume1.rebalance.files: 0 >> Volume1.rebalance.data: 0Bytes >> [Volume1.options] >> nfs.disable: on >> performance.readdir-ahead: on >> transport.address-family: inet >> features.uss: on >> >> Volume2.name: tv2 >> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >> Volume2.type: Distribute >> ......... >> ......... >> >> >> This would require minor changes to tendrl/gluster-integration >> >> definition files and code. I will draw up a spec on Tendrl/specifications >> to document the changes required. Please go ahead with your patch >> >> Thanks >> >> On Tue, Nov 22, 2016 at 9:18 AM, Atin Mukherjee >> >> wrote: >> >> We are awaiting final confirmation from Rohan. Samikshan has kept the >> changes ready and will push it to gerrit once we hear from Rohan. >> >> On Mon, Nov 21, 2016 at 12:54 PM, Shubhendu Tripathi < >> >> shtripat at redhat.com> >> >> wrote: >> >> >> Looking at options, I feel option-2 would be more feasible and might >> >> not >> >> need code changes in tendrl. >> But still lets wait for the confirmation from Rohan. >> >> Regards, >> Shubhendu >> >> >> >> On 11/21/2016 12:43 PM, Atin Mukherjee wrote: >> >> >> +tendrl-devel >> >> On Mon, Nov 21, 2016 at 12:41 PM, Samikshan Bairagya < >> >> sbairagy at redhat.com >> >> wrote: >> >> Hey Rohan, >> >> So the current get-state CLI misses volume specific options in its >> output. >> Somehow I missed it while coming up with the implementation. This >> >> patch >> >> by >> Atin is a fix for that: http://review.gluster.org/#/c/15889/1. The >> following example shows how this patch would add these new data >> >> points >> >> and >> how that would change the existing format: >> >> [Volumes] >> Volume1.name: tv1 >> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >> Volume1.type: Distribute >> >> Volume1.rebalance.files: 0 >> Volume1.rebalance.data: 0Bytes >> [Volume1.options] >> nfs.disable: on >> performance.readdir-ahead: on >> transport.address-family: inet >> features.uss: on >> >> Volume2.name: tv2 >> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >> Volume2.type: Distribute >> ......... >> ......... >> >> So essentially there would be a new section for every volume that >> >> would >> >> list the option names and corresponding values. Would adding this >> >> change >> >> still keep the get-state output parseable from Tendrl POV? >> >> Or would an output like the following make more sense? Let us know. >> Thanks. >> >> [Volumes] >> Volume1.name: tv1 >> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >> Volume1.type: Distribute >> >> Volume1.rebalance.files: 0 >> Volume1.rebalance.data: 0Bytes >> Volume1.options.nfs.disable: on >> Volume1.options.performance.readdir-ahead: on >> Volume1.options.transport.address-family: inet >> Volume1.options.features.uss: on >> >> Volume2.name: tv2 >> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >> Volume2.type: Distribute >> ......... >> ......... >> >> >> ~ Samikshan >> >> >> >> _______________________________________________ >> Tendrl-devel mailing listTendrl-devel at redhat.comhttps://www.redhat.com/mailman/listinfo/tendrl-devel >> >> -- >> >> ~ Atin (atinm) >> _______________________________________________ >> Tendrl-devel mailing listTendrl-devel at redhat.comhttps://www.redhat.com/mailman/listinfo/tendrl-devel >> >> _______________________________________________ >> Tendrl-devel mailing listTendrl-devel at redhat.comhttps://www.redhat.com/mailman/listinfo/tendrl-devel >> >> >> > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From nthomas at redhat.com Wed Nov 30 07:36:01 2016 From: nthomas at redhat.com (Nishanth Thomas) Date: Wed, 30 Nov 2016 13:06:01 +0530 Subject: [Tendrl-devel] Regarding some additional data points in the state output In-Reply-To: References: <5e1609e4-cfc3-4dc3-f421-ac51f56b9d64@redhat.com> <1988c0ba-6dfb-cdb3-a70e-4e51f02830e0@redhat.com> Message-ID: I think option 2 would be a better. Atin, can you accommodate this change? Thanks, NIshanth On Wed, Nov 30, 2016 at 1:01 PM, Shubhendu Tripathi wrote: > The ini parser is a generic utility in tendrl and I prefer not to tweak it > only for this reason. > I would rather prefer option two mentioned below. > > Regards, > Shubhendu > > > On 11/30/2016 12:43 PM, Rohan Kanade wrote: > >> 2 ways to tackle this: >> >> 1) Tweak our ini parser such that, when it sees section like >> "[Volume.options]", take the "Volume.options" text and >> prefix it to the options given below until we find another >> "Volume.*" key/value. >> >> This will basically transform the output file to what it looked like as >> the >> option 1 presented by gluster folks to us >> >> 2) If this can be patched and output can be changed to look like below, we >> are good with that too. >> >> [Volumes] >> Volume1.name: tv1 >> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >> Volume1.type: Distribute >> >> Volume1.rebalance.files: 0 >> Volume1.rebalance.data: 0Bytes >> Volume1.options.nfs.disable: on >> Volume1.options.performance.readdir-ahead: on >> Volume1.options.transport.address-family: inet >> Volume1.options.features.uss: on >> Volume2.name: tv2 >> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037aVolume2.type: Distribute >> >> On Wed, Nov 30, 2016 at 9:35 AM, Shubhendu Tripathi >> wrote: >> >> Dear Rohan, >>> >>> I accept that the below format is a valid ini file format syntactically, >>> but semantically its screwing up volumes listing. >>> The "Volumes" list returns >>> >>> [Volumes] >>> Volume1.name: test-vol >>> Volume1.id: 7942d008-e300-4fd9-8af0-5a118afd8d3d >>> Volume1.type: Distribute >>> Volume1.transport_type: tcp >>> Volume1.status: Started >>> Volume1.brickcount: 1 >>> Volume1.Brick1.path: 172.17.0.2:/tmp/b1 >>> Volume1.Brick1.hostname: 172.17.0.2 >>> Volume1.Brick1.port: 49153 >>> Volume1.Brick1.rdma_port: 0 >>> Volume1.Brick1.status: Started >>> Volume1.Brick1.signedin: True >>> Volume1.snap_count: 0 >>> Volume1.stripe_count: 1 >>> Volume1.replica_count: 1 >>> Volume1.subvol_count: 1 >>> Volume1.arbiter_count: 0 >>> Volume1.disperse_count: 0 >>> Volume1.redundancy_count: 0 >>> Volume1.quorum_status: not_applicable >>> Volume1.snapd_svc.online_status: Offline >>> Volume1.snapd_svc.inited: TrueVolume1.rebalance.id: >>> 00000000-0000-0000-0000-000000000000 >>> Volume1.rebalance.status: not_started >>> Volume1.rebalance.failures: 0 >>> Volume1.rebalance.skipped: 0 >>> Volume1.rebalance.lookedup: 0 >>> Volume1.rebalance.files: 0 >>> Volume1.rebalance.data: 0Bytes >>> >>> Also the "Volume1.options" returns the volume2 details mingled within as >>> below >>> >>> >>> features.barrier: on >>> transport.address-family: inet >>> performance.readdir-ahead: on >>> nfs.disable: on >>> >>> Volume2.name: test-vol1 >>> Volume2.id: 35854708-bb72-45a5-bdbd-77c51e5ebfb9 >>> Volume2.type: Distribute >>> Volume2.transport_type: tcp >>> Volume2.status: Started >>> Volume2.brickcount: 1 >>> Volume2.Brick1.path: 172.17.0.2:/tmp/b2 >>> Volume2.Brick1.hostname: 172.17.0.2 >>> Volume2.Brick1.port: 49152 >>> Volume2.Brick1.rdma_port: 0 >>> Volume2.Brick1.status: Started >>> Volume2.Brick1.signedin: True >>> Volume2.snap_count: 0 >>> Volume2.stripe_count: 1 >>> Volume2.replica_count: 1 >>> Volume2.subvol_count: 1 >>> Volume2.arbiter_count: 0 >>> Volume2.disperse_count: 0 >>> Volume2.redundancy_count: 0 >>> Volume2.quorum_status: not_applicable >>> Volume2.snapd_svc.online_status: Offline >>> Volume2.snapd_svc.inited: TrueVolume2.rebalance.id: >>> 00000000-0000-0000-0000-000000000000 >>> Volume2.rebalance.status: not_started >>> Volume2.rebalance.failures: 0 >>> Volume2.rebalance.skipped: 0 >>> Volume2.rebalance.lookedup: 0 >>> Volume2.rebalance.files: 0 >>> Volume2.rebalance.data: 0Bytes >>> >>> Tried debugging a little the parser for this ini file and it looks like >>> sector/sections are formed based on [] brackets and anything below one >>> section (till next [] found) is treated as one section. >>> >>> Instead, the flatted structure like "Volume1.options.nfs.disable: on" >>> would have been an easier option to parse and code change tendrl side. >>> >>> At the moment I dont find a way to resolve this mingled sections and >>> handling within tendrl parser. I have tried some tweaking in parser but >>> looks like sections are formed underneath using the library for ini >>> parser. >>> >>> Comments?? >>> >>> Regards, >>> Shubhendu >>> >>> >>> >>> On 11/22/2016 08:16 PM, Rohan Kanade wrote: >>> >>> Sample: >>> START>>> >>> >>> [Global] >>> MYUUID: 6bbf8ac2-22a0-4f08-b986-fe75aea9f654 >>> op-version: 40000 >>> >>> [Global options] >>> >>> [Peers] >>> >>> [Volumes] >>> Volume1.name: test-vol >>> Volume1.id: 7942d008-e300-4fd9-8af0-5a118afd8d3d >>> Volume1.type: Distribute >>> Volume1.transport_type: tcp >>> Volume1.status: Started >>> Volume1.brickcount: 1 >>> Volume1.Brick1.path: 172.17.0.2:/tmp/b1 >>> Volume1.Brick1.hostname: 172.17.0.2 >>> Volume1.Brick1.port: 49153 >>> Volume1.Brick1.rdma_port: 0 >>> Volume1.Brick1.status: Started >>> Volume1.Brick1.signedin: True >>> Volume1.snap_count: 0 >>> Volume1.stripe_count: 1 >>> Volume1.replica_count: 1 >>> Volume1.subvol_count: 1 >>> Volume1.arbiter_count: 0 >>> Volume1.disperse_count: 0 >>> Volume1.redundancy_count: 0 >>> Volume1.quorum_status: not_applicable >>> Volume1.snapd_svc.online_status: Offline >>> Volume1.snapd_svc.inited: TrueVolume1.rebalance.id: >>> 00000000-0000-0000-0000-000000000000 >>> Volume1.rebalance.status: not_started >>> Volume1.rebalance.failures: 0 >>> Volume1.rebalance.skipped: 0 >>> Volume1.rebalance.lookedup: 0 >>> Volume1.rebalance.files: 0 >>> Volume1.rebalance.data: 0Bytes >>> [Volume1.options] >>> features.barrier: on >>> transport.address-family: inet >>> performance.readdir-ahead: on >>> nfs.disable: on >>> >>> Volume2.name: test-vol1 >>> Volume2.id: 35854708-bb72-45a5-bdbd-77c51e5ebfb9 >>> Volume2.type: Distribute >>> Volume2.transport_type: tcp >>> Volume2.status: Started >>> Volume2.brickcount: 1 >>> Volume2.Brick1.path: 172.17.0.2:/tmp/b2 >>> Volume2.Brick1.hostname: 172.17.0.2 >>> Volume2.Brick1.port: 49152 >>> Volume2.Brick1.rdma_port: 0 >>> Volume2.Brick1.status: Started >>> Volume2.Brick1.signedin: True >>> Volume2.snap_count: 0 >>> Volume2.stripe_count: 1 >>> Volume2.replica_count: 1 >>> Volume2.subvol_count: 1 >>> Volume2.arbiter_count: 0 >>> Volume2.disperse_count: 0 >>> Volume2.redundancy_count: 0 >>> Volume2.quorum_status: not_applicable >>> Volume2.snapd_svc.online_status: Offline >>> Volume2.snapd_svc.inited: TrueVolume2.rebalance.id: >>> 00000000-0000-0000-0000-000000000000 >>> Volume2.rebalance.status: not_started >>> Volume2.rebalance.failures: 0 >>> Volume2.rebalance.skipped: 0 >>> Volume2.rebalance.lookedup: 0 >>> Volume2.rebalance.files: 0 >>> Volume2.rebalance.data: 0Bytes >>> [Volume2.options] >>> transport.address-family: inet >>> performance.readdir-ahead: on >>> nfs.disable: on >>> >>> >>> [Services]svc1.name: glustershd >>> svc1.online_status: Offline >>> svc2.name: nfs >>> svc2.online_status: Offline >>> svc3.name: bitd >>> svc3.online_status: Offline >>> svc4.name: scrub >>> svc4.online_status: Offline >>> svc5.name: quotad >>> svc5.online_status: Offline >>> >>> >>> [Misc] >>> Base port: 49152 >>> Last allocated port: 49153 >>> >>> <>> >>> On Tue, Nov 22, 2016 at 2:06 PM, Rohan Kanade < >>> rkanade at redhat.com> wrote: >>> >>> Also, please provide a full state dump example with this patch included, >>> >>> easier for Tendrl devs to get started without deploying this patch >>> >>> On Tue, Nov 22, 2016 at 1:44 PM, Rohan Kanade < >>> rkanade at redhat.com> wrote: >>> >>> Id prefer the first option >>> >>> >>> [Volumes] >>> Volume1.name: tv1 >>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>> Volume1.type: Distribute >>> >>> Volume1.rebalance.files: 0 >>> Volume1.rebalance.data: 0Bytes >>> [Volume1.options] >>> nfs.disable: on >>> performance.readdir-ahead: on >>> transport.address-family: inet >>> features.uss: on >>> >>> Volume2.name: tv2 >>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>> Volume2.type: Distribute >>> ......... >>> ......... >>> >>> >>> This would require minor changes to tendrl/gluster-integration >>> >>> definition files and code. I will draw up a spec on Tendrl/specifications >>> to document the changes required. Please go ahead with your patch >>> >>> Thanks >>> >>> On Tue, Nov 22, 2016 at 9:18 AM, Atin Mukherjee < >>> amukherj at redhat.com> >>> >>> wrote: >>> >>> We are awaiting final confirmation from Rohan. Samikshan has kept the >>> changes ready and will push it to gerrit once we hear from Rohan. >>> >>> On Mon, Nov 21, 2016 at 12:54 PM, Shubhendu Tripathi < >>> >>> shtripat at redhat.com> >>> >>> wrote: >>> >>> >>> Looking at options, I feel option-2 would be more feasible and might >>> >>> not >>> >>> need code changes in tendrl. >>> But still lets wait for the confirmation from Rohan. >>> >>> Regards, >>> Shubhendu >>> >>> >>> >>> On 11/21/2016 12:43 PM, Atin Mukherjee wrote: >>> >>> >>> +tendrl-devel >>> >>> On Mon, Nov 21, 2016 at 12:41 PM, Samikshan Bairagya < >>> >>> sbairagy at redhat.com >>> >>> wrote: >>> >>> Hey Rohan, >>> >>> So the current get-state CLI misses volume specific options in its >>> output. >>> Somehow I missed it while coming up with the implementation. This >>> >>> patch >>> >>> by >>> Atin is a fix for that: http://review.gluster.org/#/c/15889/1. The >>> following example shows how this patch would add these new data >>> >>> points >>> >>> and >>> how that would change the existing format: >>> >>> [Volumes] >>> Volume1.name: tv1 >>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>> Volume1.type: Distribute >>> >>> Volume1.rebalance.files: 0 >>> Volume1.rebalance.data: 0Bytes >>> [Volume1.options] >>> nfs.disable: on >>> performance.readdir-ahead: on >>> transport.address-family: inet >>> features.uss: on >>> >>> Volume2.name: tv2 >>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>> Volume2.type: Distribute >>> ......... >>> ......... >>> >>> So essentially there would be a new section for every volume that >>> >>> would >>> >>> list the option names and corresponding values. Would adding this >>> >>> change >>> >>> still keep the get-state output parseable from Tendrl POV? >>> >>> Or would an output like the following make more sense? Let us know. >>> Thanks. >>> >>> [Volumes] >>> Volume1.name: tv1 >>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>> Volume1.type: Distribute >>> >>> Volume1.rebalance.files: 0 >>> Volume1.rebalance.data: 0Bytes >>> Volume1.options.nfs.disable: on >>> Volume1.options.performance.readdir-ahead: on >>> Volume1.options.transport.address-family: inet >>> Volume1.options.features.uss: on >>> >>> Volume2.name: tv2 >>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>> Volume2.type: Distribute >>> ......... >>> ......... >>> >>> >>> ~ Samikshan >>> >>> >>> >>> _______________________________________________ >>> Tendrl-devel mailing listTendrl-devel at redhat.comhttps:// >>> www.redhat.com/mailman/listinfo/tendrl-devel >>> >>> -- >>> >>> ~ Atin (atinm) >>> _______________________________________________ >>> Tendrl-devel mailing listTendrl-devel at redhat.comhttps:// >>> www.redhat.com/mailman/listinfo/tendrl-devel >>> >>> _______________________________________________ >>> Tendrl-devel mailing listTendrl-devel at redhat.comhttps:// >>> www.redhat.com/mailman/listinfo/tendrl-devel >>> >>> >>> >>> _______________________________________________ >> Tendrl-devel mailing list >> Tendrl-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/tendrl-devel >> > > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From sbairagy at redhat.com Wed Nov 30 09:17:29 2016 From: sbairagy at redhat.com (Samikshan Bairagya) Date: Wed, 30 Nov 2016 14:47:29 +0530 Subject: [Tendrl-devel] Regarding some additional data points in the state output In-Reply-To: References: <5e1609e4-cfc3-4dc3-f421-ac51f56b9d64@redhat.com> <1988c0ba-6dfb-cdb3-a70e-4e51f02830e0@redhat.com> Message-ID: <6a77b36c-086a-8903-e5eb-b277a45a178f@redhat.com> On 11/30/2016 09:35 AM, Shubhendu Tripathi wrote: > Dear Rohan, > > I accept that the below format is a valid ini file format syntactically, > but semantically its screwing up volumes listing. > The "Volumes" list returns > > [Volumes] > Volume1.name: test-vol > Volume1.id: 7942d008-e300-4fd9-8af0-5a118afd8d3d > Volume1.type: Distribute > Volume1.transport_type: tcp > Volume1.status: Started > Volume1.brickcount: 1 > Volume1.Brick1.path: 172.17.0.2:/tmp/b1 > Volume1.Brick1.hostname: 172.17.0.2 > Volume1.Brick1.port: 49153 > Volume1.Brick1.rdma_port: 0 > Volume1.Brick1.status: Started > Volume1.Brick1.signedin: True > Volume1.snap_count: 0 > Volume1.stripe_count: 1 > Volume1.replica_count: 1 > Volume1.subvol_count: 1 > Volume1.arbiter_count: 0 > Volume1.disperse_count: 0 > Volume1.redundancy_count: 0 > Volume1.quorum_status: not_applicable > Volume1.snapd_svc.online_status: Offline > Volume1.snapd_svc.inited: True > Volume1.rebalance.id: 00000000-0000-0000-0000-000000000000 > Volume1.rebalance.status: not_started > Volume1.rebalance.failures: 0 > Volume1.rebalance.skipped: 0 > Volume1.rebalance.lookedup: 0 > Volume1.rebalance.files: 0 > Volume1.rebalance.data: 0Bytes > > Also the "Volume1.options" returns the volume2 details mingled within as > below > > features.barrier: on > transport.address-family: inet > performance.readdir-ahead: on > nfs.disable: on > > Volume2.name: test-vol1 > Volume2.id: 35854708-bb72-45a5-bdbd-77c51e5ebfb9 > Volume2.type: Distribute > Volume2.transport_type: tcp > Volume2.status: Started > Volume2.brickcount: 1 > Volume2.Brick1.path: 172.17.0.2:/tmp/b2 > Volume2.Brick1.hostname: 172.17.0.2 > Volume2.Brick1.port: 49152 > Volume2.Brick1.rdma_port: 0 > Volume2.Brick1.status: Started > Volume2.Brick1.signedin: True > Volume2.snap_count: 0 > Volume2.stripe_count: 1 > Volume2.replica_count: 1 > Volume2.subvol_count: 1 > Volume2.arbiter_count: 0 > Volume2.disperse_count: 0 > Volume2.redundancy_count: 0 > Volume2.quorum_status: not_applicable > Volume2.snapd_svc.online_status: Offline > Volume2.snapd_svc.inited: True > Volume2.rebalance.id: 00000000-0000-0000-0000-000000000000 > Volume2.rebalance.status: not_started > Volume2.rebalance.failures: 0 > Volume2.rebalance.skipped: 0 > Volume2.rebalance.lookedup: 0 > Volume2.rebalance.files: 0 > Volume2.rebalance.data: 0Bytes > > Tried debugging a little the parser for this ini file and it looks like > sector/sections are formed based on [] brackets and anything below one > section (till next [] found) is treated as one section. > > Instead, the flatted structure like "Volume1.options.nfs.disable: on" > would have been an easier option to parse and code change tendrl side. > Hi, A patch for this is ready here: http://review.gluster.org/15975. Thanks. ~ Samikshan > At the moment I dont find a way to resolve this mingled sections and > handling within tendrl parser. I have tried some tweaking in parser but > looks like sections are formed underneath using the library for ini parser. > > Comments?? > > Regards, > Shubhendu > > > On 11/22/2016 08:16 PM, Rohan Kanade wrote: >> Sample: >> START>>> >> >> [Global] >> MYUUID: 6bbf8ac2-22a0-4f08-b986-fe75aea9f654 >> op-version: 40000 >> >> [Global options] >> >> [Peers] >> >> [Volumes] >> Volume1.name: test-vol >> Volume1.id: 7942d008-e300-4fd9-8af0-5a118afd8d3d >> Volume1.type: Distribute >> Volume1.transport_type: tcp >> Volume1.status: Started >> Volume1.brickcount: 1 >> Volume1.Brick1.path: 172.17.0.2:/tmp/b1 >> Volume1.Brick1.hostname: 172.17.0.2 >> Volume1.Brick1.port: 49153 >> Volume1.Brick1.rdma_port: 0 >> Volume1.Brick1.status: Started >> Volume1.Brick1.signedin: True >> Volume1.snap_count: 0 >> Volume1.stripe_count: 1 >> Volume1.replica_count: 1 >> Volume1.subvol_count: 1 >> Volume1.arbiter_count: 0 >> Volume1.disperse_count: 0 >> Volume1.redundancy_count: 0 >> Volume1.quorum_status: not_applicable >> Volume1.snapd_svc.online_status: Offline >> Volume1.snapd_svc.inited: True >> Volume1.rebalance.id: 00000000-0000-0000-0000-000000000000 >> Volume1.rebalance.status: not_started >> Volume1.rebalance.failures: 0 >> Volume1.rebalance.skipped: 0 >> Volume1.rebalance.lookedup: 0 >> Volume1.rebalance.files: 0 >> Volume1.rebalance.data: 0Bytes >> [Volume1.options] >> features.barrier: on >> transport.address-family: inet >> performance.readdir-ahead: on >> nfs.disable: on >> >> Volume2.name: test-vol1 >> Volume2.id: 35854708-bb72-45a5-bdbd-77c51e5ebfb9 >> Volume2.type: Distribute >> Volume2.transport_type: tcp >> Volume2.status: Started >> Volume2.brickcount: 1 >> Volume2.Brick1.path: 172.17.0.2:/tmp/b2 >> Volume2.Brick1.hostname: 172.17.0.2 >> Volume2.Brick1.port: 49152 >> Volume2.Brick1.rdma_port: 0 >> Volume2.Brick1.status: Started >> Volume2.Brick1.signedin: True >> Volume2.snap_count: 0 >> Volume2.stripe_count: 1 >> Volume2.replica_count: 1 >> Volume2.subvol_count: 1 >> Volume2.arbiter_count: 0 >> Volume2.disperse_count: 0 >> Volume2.redundancy_count: 0 >> Volume2.quorum_status: not_applicable >> Volume2.snapd_svc.online_status: Offline >> Volume2.snapd_svc.inited: True >> Volume2.rebalance.id: 00000000-0000-0000-0000-000000000000 >> Volume2.rebalance.status: not_started >> Volume2.rebalance.failures: 0 >> Volume2.rebalance.skipped: 0 >> Volume2.rebalance.lookedup: 0 >> Volume2.rebalance.files: 0 >> Volume2.rebalance.data: 0Bytes >> [Volume2.options] >> transport.address-family: inet >> performance.readdir-ahead: on >> nfs.disable: on >> >> >> [Services] >> svc1.name: glustershd >> svc1.online_status: Offline >> >> svc2.name: nfs >> svc2.online_status: Offline >> >> svc3.name: bitd >> svc3.online_status: Offline >> >> svc4.name: scrub >> svc4.online_status: Offline >> >> svc5.name: quotad >> svc5.online_status: Offline >> >> >> [Misc] >> Base port: 49152 >> Last allocated port: 49153 >> >> <> >> On Tue, Nov 22, 2016 at 2:06 PM, Rohan Kanade wrote: >>> Also, please provide a full state dump example with this patch included, >> easier for Tendrl devs to get started without deploying this patch >>> On Tue, Nov 22, 2016 at 1:44 PM, Rohan Kanade >>> wrote: >>>> Id prefer the first option >>>> >>>> >>>> [Volumes] >>>> Volume1.name: tv1 >>>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>>> Volume1.type: Distribute >>>> >>>> Volume1.rebalance.files: 0 >>>> Volume1.rebalance.data: 0Bytes >>>> [Volume1.options] >>>> nfs.disable: on >>>> performance.readdir-ahead: on >>>> transport.address-family: inet >>>> features.uss: on >>>> >>>> Volume2.name: tv2 >>>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>>> Volume2.type: Distribute >>>> ......... >>>> ......... >>>> >>>> >>>> This would require minor changes to tendrl/gluster-integration >> definition files and code. I will draw up a spec on Tendrl/specifications >> to document the changes required. Please go ahead with your patch >>>> Thanks >>>> >>>> On Tue, Nov 22, 2016 at 9:18 AM, Atin Mukherjee >> wrote: >>>>> We are awaiting final confirmation from Rohan. Samikshan has kept the >>>>> changes ready and will push it to gerrit once we hear from Rohan. >>>>> >>>>> On Mon, Nov 21, 2016 at 12:54 PM, Shubhendu Tripathi < >> shtripat at redhat.com> >>>>> wrote: >>>>> >>>>>> Looking at options, I feel option-2 would be more feasible and might >> not >>>>>> need code changes in tendrl. >>>>>> But still lets wait for the confirmation from Rohan. >>>>>> >>>>>> Regards, >>>>>> Shubhendu >>>>>> >>>>>> >>>>>> >>>>>> On 11/21/2016 12:43 PM, Atin Mukherjee wrote: >>>>>> >>>>>>> +tendrl-devel >>>>>>> >>>>>>> On Mon, Nov 21, 2016 at 12:41 PM, Samikshan Bairagya < >> sbairagy at redhat.com >>>>>>> wrote: >>>>>>> >>>>>>> Hey Rohan, >>>>>>>> So the current get-state CLI misses volume specific options in its >>>>>>>> output. >>>>>>>> Somehow I missed it while coming up with the implementation. This >> patch >>>>>>>> by >>>>>>>> Atin is a fix for that: http://review.gluster.org/#/c/15889/1. The >>>>>>>> following example shows how this patch would add these new data >> points >>>>>>>> and >>>>>>>> how that would change the existing format: >>>>>>>> >>>>>>>> [Volumes] >>>>>>>> Volume1.name: tv1 >>>>>>>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>>>>>>> Volume1.type: Distribute >>>>>>>> >>>>>>>> Volume1.rebalance.files: 0 >>>>>>>> Volume1.rebalance.data: 0Bytes >>>>>>>> [Volume1.options] >>>>>>>> nfs.disable: on >>>>>>>> performance.readdir-ahead: on >>>>>>>> transport.address-family: inet >>>>>>>> features.uss: on >>>>>>>> >>>>>>>> Volume2.name: tv2 >>>>>>>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>>>>>>> Volume2.type: Distribute >>>>>>>> ......... >>>>>>>> ......... >>>>>>>> >>>>>>>> So essentially there would be a new section for every volume that >> would >>>>>>>> list the option names and corresponding values. Would adding this >> change >>>>>>>> still keep the get-state output parseable from Tendrl POV? >>>>>>>> >>>>>>>> Or would an output like the following make more sense? Let us know. >>>>>>>> Thanks. >>>>>>>> >>>>>>>> [Volumes] >>>>>>>> Volume1.name: tv1 >>>>>>>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>>>>>>> Volume1.type: Distribute >>>>>>>> >>>>>>>> Volume1.rebalance.files: 0 >>>>>>>> Volume1.rebalance.data: 0Bytes >>>>>>>> Volume1.options.nfs.disable: on >>>>>>>> Volume1.options.performance.readdir-ahead: on >>>>>>>> Volume1.options.transport.address-family: inet >>>>>>>> Volume1.options.features.uss: on >>>>>>>> >>>>>>>> Volume2.name: tv2 >>>>>>>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>>>>>>> Volume2.type: Distribute >>>>>>>> ......... >>>>>>>> ......... >>>>>>>> >>>>>>>> >>>>>>>> ~ Samikshan >>>>>>>> >>>>>>>> >>>>>>> >>>>>> _______________________________________________ >>>>>> Tendrl-devel mailing list >>>>>> Tendrl-devel at redhat.com >>>>>> https://www.redhat.com/mailman/listinfo/tendrl-devel >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> ~ Atin (atinm) >>>>> _______________________________________________ >>>>> Tendrl-devel mailing list >>>>> Tendrl-devel at redhat.com >>>>> https://www.redhat.com/mailman/listinfo/tendrl-devel >>>> >> _______________________________________________ >> Tendrl-devel mailing list >> Tendrl-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/tendrl-devel > > From rkanade at redhat.com Wed Nov 30 09:23:19 2016 From: rkanade at redhat.com (Rohan Kanade) Date: Wed, 30 Nov 2016 14:53:19 +0530 Subject: [Tendrl-devel] Regarding some additional data points in the state output In-Reply-To: <6a77b36c-086a-8903-e5eb-b277a45a178f@redhat.com> References: <5e1609e4-cfc3-4dc3-f421-ac51f56b9d64@redhat.com> <1988c0ba-6dfb-cdb3-a70e-4e51f02830e0@redhat.com> <6a77b36c-086a-8903-e5eb-b277a45a178f@redhat.com> Message-ID: @samikshan, Please provide sample output of the get-state cli based on this patch On Wed, Nov 30, 2016 at 2:47 PM, Samikshan Bairagya wrote: > > > On 11/30/2016 09:35 AM, Shubhendu Tripathi wrote: > >> Dear Rohan, >> >> I accept that the below format is a valid ini file format syntactically, >> but semantically its screwing up volumes listing. >> The "Volumes" list returns >> >> [Volumes] >> Volume1.name: test-vol >> Volume1.id: 7942d008-e300-4fd9-8af0-5a118afd8d3d >> Volume1.type: Distribute >> Volume1.transport_type: tcp >> Volume1.status: Started >> Volume1.brickcount: 1 >> Volume1.Brick1.path: 172.17.0.2:/tmp/b1 >> Volume1.Brick1.hostname: 172.17.0.2 >> Volume1.Brick1.port: 49153 >> Volume1.Brick1.rdma_port: 0 >> Volume1.Brick1.status: Started >> Volume1.Brick1.signedin: True >> Volume1.snap_count: 0 >> Volume1.stripe_count: 1 >> Volume1.replica_count: 1 >> Volume1.subvol_count: 1 >> Volume1.arbiter_count: 0 >> Volume1.disperse_count: 0 >> Volume1.redundancy_count: 0 >> Volume1.quorum_status: not_applicable >> Volume1.snapd_svc.online_status: Offline >> Volume1.snapd_svc.inited: True >> Volume1.rebalance.id: 00000000-0000-0000-0000-000000000000 >> Volume1.rebalance.status: not_started >> Volume1.rebalance.failures: 0 >> Volume1.rebalance.skipped: 0 >> Volume1.rebalance.lookedup: 0 >> Volume1.rebalance.files: 0 >> Volume1.rebalance.data: 0Bytes >> >> Also the "Volume1.options" returns the volume2 details mingled within as >> below >> >> features.barrier: on >> transport.address-family: inet >> performance.readdir-ahead: on >> nfs.disable: on >> >> Volume2.name: test-vol1 >> Volume2.id: 35854708-bb72-45a5-bdbd-77c51e5ebfb9 >> Volume2.type: Distribute >> Volume2.transport_type: tcp >> Volume2.status: Started >> Volume2.brickcount: 1 >> Volume2.Brick1.path: 172.17.0.2:/tmp/b2 >> Volume2.Brick1.hostname: 172.17.0.2 >> Volume2.Brick1.port: 49152 >> Volume2.Brick1.rdma_port: 0 >> Volume2.Brick1.status: Started >> Volume2.Brick1.signedin: True >> Volume2.snap_count: 0 >> Volume2.stripe_count: 1 >> Volume2.replica_count: 1 >> Volume2.subvol_count: 1 >> Volume2.arbiter_count: 0 >> Volume2.disperse_count: 0 >> Volume2.redundancy_count: 0 >> Volume2.quorum_status: not_applicable >> Volume2.snapd_svc.online_status: Offline >> Volume2.snapd_svc.inited: True >> Volume2.rebalance.id: 00000000-0000-0000-0000-000000000000 >> Volume2.rebalance.status: not_started >> Volume2.rebalance.failures: 0 >> Volume2.rebalance.skipped: 0 >> Volume2.rebalance.lookedup: 0 >> Volume2.rebalance.files: 0 >> Volume2.rebalance.data: 0Bytes >> >> Tried debugging a little the parser for this ini file and it looks like >> sector/sections are formed based on [] brackets and anything below one >> section (till next [] found) is treated as one section. >> >> Instead, the flatted structure like "Volume1.options.nfs.disable: on" >> would have been an easier option to parse and code change tendrl side. >> >> > Hi, A patch for this is ready here: http://review.gluster.org/15975. > Thanks. > > ~ Samikshan > > > > At the moment I dont find a way to resolve this mingled sections and >> handling within tendrl parser. I have tried some tweaking in parser but >> looks like sections are formed underneath using the library for ini >> parser. >> >> Comments?? >> >> Regards, >> Shubhendu >> >> >> On 11/22/2016 08:16 PM, Rohan Kanade wrote: >> >>> Sample: >>> START>>> >>> >>> [Global] >>> MYUUID: 6bbf8ac2-22a0-4f08-b986-fe75aea9f654 >>> op-version: 40000 >>> >>> [Global options] >>> >>> [Peers] >>> >>> [Volumes] >>> Volume1.name: test-vol >>> Volume1.id: 7942d008-e300-4fd9-8af0-5a118afd8d3d >>> Volume1.type: Distribute >>> Volume1.transport_type: tcp >>> Volume1.status: Started >>> Volume1.brickcount: 1 >>> Volume1.Brick1.path: 172.17.0.2:/tmp/b1 >>> Volume1.Brick1.hostname: 172.17.0.2 >>> Volume1.Brick1.port: 49153 >>> Volume1.Brick1.rdma_port: 0 >>> Volume1.Brick1.status: Started >>> Volume1.Brick1.signedin: True >>> Volume1.snap_count: 0 >>> Volume1.stripe_count: 1 >>> Volume1.replica_count: 1 >>> Volume1.subvol_count: 1 >>> Volume1.arbiter_count: 0 >>> Volume1.disperse_count: 0 >>> Volume1.redundancy_count: 0 >>> Volume1.quorum_status: not_applicable >>> Volume1.snapd_svc.online_status: Offline >>> Volume1.snapd_svc.inited: True >>> Volume1.rebalance.id: 00000000-0000-0000-0000-000000000000 >>> Volume1.rebalance.status: not_started >>> Volume1.rebalance.failures: 0 >>> Volume1.rebalance.skipped: 0 >>> Volume1.rebalance.lookedup: 0 >>> Volume1.rebalance.files: 0 >>> Volume1.rebalance.data: 0Bytes >>> [Volume1.options] >>> features.barrier: on >>> transport.address-family: inet >>> performance.readdir-ahead: on >>> nfs.disable: on >>> >>> Volume2.name: test-vol1 >>> Volume2.id: 35854708-bb72-45a5-bdbd-77c51e5ebfb9 >>> Volume2.type: Distribute >>> Volume2.transport_type: tcp >>> Volume2.status: Started >>> Volume2.brickcount: 1 >>> Volume2.Brick1.path: 172.17.0.2:/tmp/b2 >>> Volume2.Brick1.hostname: 172.17.0.2 >>> Volume2.Brick1.port: 49152 >>> Volume2.Brick1.rdma_port: 0 >>> Volume2.Brick1.status: Started >>> Volume2.Brick1.signedin: True >>> Volume2.snap_count: 0 >>> Volume2.stripe_count: 1 >>> Volume2.replica_count: 1 >>> Volume2.subvol_count: 1 >>> Volume2.arbiter_count: 0 >>> Volume2.disperse_count: 0 >>> Volume2.redundancy_count: 0 >>> Volume2.quorum_status: not_applicable >>> Volume2.snapd_svc.online_status: Offline >>> Volume2.snapd_svc.inited: True >>> Volume2.rebalance.id: 00000000-0000-0000-0000-000000000000 >>> Volume2.rebalance.status: not_started >>> Volume2.rebalance.failures: 0 >>> Volume2.rebalance.skipped: 0 >>> Volume2.rebalance.lookedup: 0 >>> Volume2.rebalance.files: 0 >>> Volume2.rebalance.data: 0Bytes >>> [Volume2.options] >>> transport.address-family: inet >>> performance.readdir-ahead: on >>> nfs.disable: on >>> >>> >>> [Services] >>> svc1.name: glustershd >>> svc1.online_status: Offline >>> >>> svc2.name: nfs >>> svc2.online_status: Offline >>> >>> svc3.name: bitd >>> svc3.online_status: Offline >>> >>> svc4.name: scrub >>> svc4.online_status: Offline >>> >>> svc5.name: quotad >>> svc5.online_status: Offline >>> >>> >>> [Misc] >>> Base port: 49152 >>> Last allocated port: 49153 >>> >>> <>> >>> On Tue, Nov 22, 2016 at 2:06 PM, Rohan Kanade >>> wrote: >>> >>>> Also, please provide a full state dump example with this patch included, >>>> >>> easier for Tendrl devs to get started without deploying this patch >>> >>>> On Tue, Nov 22, 2016 at 1:44 PM, Rohan Kanade >>>> wrote: >>>> >>>>> Id prefer the first option >>>>> >>>>> >>>>> [Volumes] >>>>> Volume1.name: tv1 >>>>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>>>> Volume1.type: Distribute >>>>> >>>>> Volume1.rebalance.files: 0 >>>>> Volume1.rebalance.data: 0Bytes >>>>> [Volume1.options] >>>>> nfs.disable: on >>>>> performance.readdir-ahead: on >>>>> transport.address-family: inet >>>>> features.uss: on >>>>> >>>>> Volume2.name: tv2 >>>>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>>>> Volume2.type: Distribute >>>>> ......... >>>>> ......... >>>>> >>>>> >>>>> This would require minor changes to tendrl/gluster-integration >>>>> >>>> definition files and code. I will draw up a spec on >>> Tendrl/specifications >>> to document the changes required. Please go ahead with your patch >>> >>>> Thanks >>>>> >>>>> On Tue, Nov 22, 2016 at 9:18 AM, Atin Mukherjee >>>>> >>>> wrote: >>> >>>> We are awaiting final confirmation from Rohan. Samikshan has kept the >>>>>> changes ready and will push it to gerrit once we hear from Rohan. >>>>>> >>>>>> On Mon, Nov 21, 2016 at 12:54 PM, Shubhendu Tripathi < >>>>>> >>>>> shtripat at redhat.com> >>> >>>> wrote: >>>>>> >>>>>> Looking at options, I feel option-2 would be more feasible and might >>>>>>> >>>>>> not >>> >>>> need code changes in tendrl. >>>>>>> But still lets wait for the confirmation from Rohan. >>>>>>> >>>>>>> Regards, >>>>>>> Shubhendu >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 11/21/2016 12:43 PM, Atin Mukherjee wrote: >>>>>>> >>>>>>> +tendrl-devel >>>>>>>> >>>>>>>> On Mon, Nov 21, 2016 at 12:41 PM, Samikshan Bairagya < >>>>>>>> >>>>>>> sbairagy at redhat.com >>> >>>> wrote: >>>>>>>> >>>>>>>> Hey Rohan, >>>>>>>> >>>>>>>>> So the current get-state CLI misses volume specific options in its >>>>>>>>> output. >>>>>>>>> Somehow I missed it while coming up with the implementation. This >>>>>>>>> >>>>>>>> patch >>> >>>> by >>>>>>>>> Atin is a fix for that: http://review.gluster.org/#/c/15889/1. The >>>>>>>>> following example shows how this patch would add these new data >>>>>>>>> >>>>>>>> points >>> >>>> and >>>>>>>>> how that would change the existing format: >>>>>>>>> >>>>>>>>> [Volumes] >>>>>>>>> Volume1.name: tv1 >>>>>>>>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>>>>>>>> Volume1.type: Distribute >>>>>>>>> >>>>>>>>> Volume1.rebalance.files: 0 >>>>>>>>> Volume1.rebalance.data: 0Bytes >>>>>>>>> [Volume1.options] >>>>>>>>> nfs.disable: on >>>>>>>>> performance.readdir-ahead: on >>>>>>>>> transport.address-family: inet >>>>>>>>> features.uss: on >>>>>>>>> >>>>>>>>> Volume2.name: tv2 >>>>>>>>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>>>>>>>> Volume2.type: Distribute >>>>>>>>> ......... >>>>>>>>> ......... >>>>>>>>> >>>>>>>>> So essentially there would be a new section for every volume that >>>>>>>>> >>>>>>>> would >>> >>>> list the option names and corresponding values. Would adding this >>>>>>>>> >>>>>>>> change >>> >>>> still keep the get-state output parseable from Tendrl POV? >>>>>>>>> >>>>>>>>> Or would an output like the following make more sense? Let us know. >>>>>>>>> Thanks. >>>>>>>>> >>>>>>>>> [Volumes] >>>>>>>>> Volume1.name: tv1 >>>>>>>>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>>>>>>>> Volume1.type: Distribute >>>>>>>>> >>>>>>>>> Volume1.rebalance.files: 0 >>>>>>>>> Volume1.rebalance.data: 0Bytes >>>>>>>>> Volume1.options.nfs.disable: on >>>>>>>>> Volume1.options.performance.readdir-ahead: on >>>>>>>>> Volume1.options.transport.address-family: inet >>>>>>>>> Volume1.options.features.uss: on >>>>>>>>> >>>>>>>>> Volume2.name: tv2 >>>>>>>>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>>>>>>>> Volume2.type: Distribute >>>>>>>>> ......... >>>>>>>>> ......... >>>>>>>>> >>>>>>>>> >>>>>>>>> ~ Samikshan >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> _______________________________________________ >>>>>>> Tendrl-devel mailing list >>>>>>> Tendrl-devel at redhat.com >>>>>>> https://www.redhat.com/mailman/listinfo/tendrl-devel >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> ~ Atin (atinm) >>>>>> _______________________________________________ >>>>>> Tendrl-devel mailing list >>>>>> Tendrl-devel at redhat.com >>>>>> https://www.redhat.com/mailman/listinfo/tendrl-devel >>>>>> >>>>> >>>>> _______________________________________________ >>> Tendrl-devel mailing list >>> Tendrl-devel at redhat.com >>> https://www.redhat.com/mailman/listinfo/tendrl-devel >>> >> >> >> From sbairagy at redhat.com Wed Nov 30 09:43:16 2016 From: sbairagy at redhat.com (Samikshan Bairagya) Date: Wed, 30 Nov 2016 15:13:16 +0530 Subject: [Tendrl-devel] Regarding some additional data points in the state output In-Reply-To: References: <5e1609e4-cfc3-4dc3-f421-ac51f56b9d64@redhat.com> <1988c0ba-6dfb-cdb3-a70e-4e51f02830e0@redhat.com> <6a77b36c-086a-8903-e5eb-b277a45a178f@redhat.com> Message-ID: <040d4c4e-c468-4ae0-81dd-3b4fa5238c4c@redhat.com> On 11/30/2016 02:53 PM, Rohan Kanade wrote: > @samikshan, > > Please provide sample output of the get-state cli based on this patch > A sample output file is attached with this email. ~ Samikshan > On Wed, Nov 30, 2016 at 2:47 PM, Samikshan Bairagya > wrote: > >> >> >> On 11/30/2016 09:35 AM, Shubhendu Tripathi wrote: >> >>> Dear Rohan, >>> >>> I accept that the below format is a valid ini file format syntactically, >>> but semantically its screwing up volumes listing. >>> The "Volumes" list returns >>> >>> [Volumes] >>> Volume1.name: test-vol >>> Volume1.id: 7942d008-e300-4fd9-8af0-5a118afd8d3d >>> Volume1.type: Distribute >>> Volume1.transport_type: tcp >>> Volume1.status: Started >>> Volume1.brickcount: 1 >>> Volume1.Brick1.path: 172.17.0.2:/tmp/b1 >>> Volume1.Brick1.hostname: 172.17.0.2 >>> Volume1.Brick1.port: 49153 >>> Volume1.Brick1.rdma_port: 0 >>> Volume1.Brick1.status: Started >>> Volume1.Brick1.signedin: True >>> Volume1.snap_count: 0 >>> Volume1.stripe_count: 1 >>> Volume1.replica_count: 1 >>> Volume1.subvol_count: 1 >>> Volume1.arbiter_count: 0 >>> Volume1.disperse_count: 0 >>> Volume1.redundancy_count: 0 >>> Volume1.quorum_status: not_applicable >>> Volume1.snapd_svc.online_status: Offline >>> Volume1.snapd_svc.inited: True >>> Volume1.rebalance.id: 00000000-0000-0000-0000-000000000000 >>> Volume1.rebalance.status: not_started >>> Volume1.rebalance.failures: 0 >>> Volume1.rebalance.skipped: 0 >>> Volume1.rebalance.lookedup: 0 >>> Volume1.rebalance.files: 0 >>> Volume1.rebalance.data: 0Bytes >>> >>> Also the "Volume1.options" returns the volume2 details mingled within as >>> below >>> >>> features.barrier: on >>> transport.address-family: inet >>> performance.readdir-ahead: on >>> nfs.disable: on >>> >>> Volume2.name: test-vol1 >>> Volume2.id: 35854708-bb72-45a5-bdbd-77c51e5ebfb9 >>> Volume2.type: Distribute >>> Volume2.transport_type: tcp >>> Volume2.status: Started >>> Volume2.brickcount: 1 >>> Volume2.Brick1.path: 172.17.0.2:/tmp/b2 >>> Volume2.Brick1.hostname: 172.17.0.2 >>> Volume2.Brick1.port: 49152 >>> Volume2.Brick1.rdma_port: 0 >>> Volume2.Brick1.status: Started >>> Volume2.Brick1.signedin: True >>> Volume2.snap_count: 0 >>> Volume2.stripe_count: 1 >>> Volume2.replica_count: 1 >>> Volume2.subvol_count: 1 >>> Volume2.arbiter_count: 0 >>> Volume2.disperse_count: 0 >>> Volume2.redundancy_count: 0 >>> Volume2.quorum_status: not_applicable >>> Volume2.snapd_svc.online_status: Offline >>> Volume2.snapd_svc.inited: True >>> Volume2.rebalance.id: 00000000-0000-0000-0000-000000000000 >>> Volume2.rebalance.status: not_started >>> Volume2.rebalance.failures: 0 >>> Volume2.rebalance.skipped: 0 >>> Volume2.rebalance.lookedup: 0 >>> Volume2.rebalance.files: 0 >>> Volume2.rebalance.data: 0Bytes >>> >>> Tried debugging a little the parser for this ini file and it looks like >>> sector/sections are formed based on [] brackets and anything below one >>> section (till next [] found) is treated as one section. >>> >>> Instead, the flatted structure like "Volume1.options.nfs.disable: on" >>> would have been an easier option to parse and code change tendrl side. >>> >>> >> Hi, A patch for this is ready here: http://review.gluster.org/15975. >> Thanks. >> >> ~ Samikshan >> >> >> >> At the moment I dont find a way to resolve this mingled sections and >>> handling within tendrl parser. I have tried some tweaking in parser but >>> looks like sections are formed underneath using the library for ini >>> parser. >>> >>> Comments?? >>> >>> Regards, >>> Shubhendu >>> >>> >>> On 11/22/2016 08:16 PM, Rohan Kanade wrote: >>> >>>> Sample: >>>> START>>> >>>> >>>> [Global] >>>> MYUUID: 6bbf8ac2-22a0-4f08-b986-fe75aea9f654 >>>> op-version: 40000 >>>> >>>> [Global options] >>>> >>>> [Peers] >>>> >>>> [Volumes] >>>> Volume1.name: test-vol >>>> Volume1.id: 7942d008-e300-4fd9-8af0-5a118afd8d3d >>>> Volume1.type: Distribute >>>> Volume1.transport_type: tcp >>>> Volume1.status: Started >>>> Volume1.brickcount: 1 >>>> Volume1.Brick1.path: 172.17.0.2:/tmp/b1 >>>> Volume1.Brick1.hostname: 172.17.0.2 >>>> Volume1.Brick1.port: 49153 >>>> Volume1.Brick1.rdma_port: 0 >>>> Volume1.Brick1.status: Started >>>> Volume1.Brick1.signedin: True >>>> Volume1.snap_count: 0 >>>> Volume1.stripe_count: 1 >>>> Volume1.replica_count: 1 >>>> Volume1.subvol_count: 1 >>>> Volume1.arbiter_count: 0 >>>> Volume1.disperse_count: 0 >>>> Volume1.redundancy_count: 0 >>>> Volume1.quorum_status: not_applicable >>>> Volume1.snapd_svc.online_status: Offline >>>> Volume1.snapd_svc.inited: True >>>> Volume1.rebalance.id: 00000000-0000-0000-0000-000000000000 >>>> Volume1.rebalance.status: not_started >>>> Volume1.rebalance.failures: 0 >>>> Volume1.rebalance.skipped: 0 >>>> Volume1.rebalance.lookedup: 0 >>>> Volume1.rebalance.files: 0 >>>> Volume1.rebalance.data: 0Bytes >>>> [Volume1.options] >>>> features.barrier: on >>>> transport.address-family: inet >>>> performance.readdir-ahead: on >>>> nfs.disable: on >>>> >>>> Volume2.name: test-vol1 >>>> Volume2.id: 35854708-bb72-45a5-bdbd-77c51e5ebfb9 >>>> Volume2.type: Distribute >>>> Volume2.transport_type: tcp >>>> Volume2.status: Started >>>> Volume2.brickcount: 1 >>>> Volume2.Brick1.path: 172.17.0.2:/tmp/b2 >>>> Volume2.Brick1.hostname: 172.17.0.2 >>>> Volume2.Brick1.port: 49152 >>>> Volume2.Brick1.rdma_port: 0 >>>> Volume2.Brick1.status: Started >>>> Volume2.Brick1.signedin: True >>>> Volume2.snap_count: 0 >>>> Volume2.stripe_count: 1 >>>> Volume2.replica_count: 1 >>>> Volume2.subvol_count: 1 >>>> Volume2.arbiter_count: 0 >>>> Volume2.disperse_count: 0 >>>> Volume2.redundancy_count: 0 >>>> Volume2.quorum_status: not_applicable >>>> Volume2.snapd_svc.online_status: Offline >>>> Volume2.snapd_svc.inited: True >>>> Volume2.rebalance.id: 00000000-0000-0000-0000-000000000000 >>>> Volume2.rebalance.status: not_started >>>> Volume2.rebalance.failures: 0 >>>> Volume2.rebalance.skipped: 0 >>>> Volume2.rebalance.lookedup: 0 >>>> Volume2.rebalance.files: 0 >>>> Volume2.rebalance.data: 0Bytes >>>> [Volume2.options] >>>> transport.address-family: inet >>>> performance.readdir-ahead: on >>>> nfs.disable: on >>>> >>>> >>>> [Services] >>>> svc1.name: glustershd >>>> svc1.online_status: Offline >>>> >>>> svc2.name: nfs >>>> svc2.online_status: Offline >>>> >>>> svc3.name: bitd >>>> svc3.online_status: Offline >>>> >>>> svc4.name: scrub >>>> svc4.online_status: Offline >>>> >>>> svc5.name: quotad >>>> svc5.online_status: Offline >>>> >>>> >>>> [Misc] >>>> Base port: 49152 >>>> Last allocated port: 49153 >>>> >>>> <>>> >>>> On Tue, Nov 22, 2016 at 2:06 PM, Rohan Kanade >>>> wrote: >>>> >>>>> Also, please provide a full state dump example with this patch included, >>>>> >>>> easier for Tendrl devs to get started without deploying this patch >>>> >>>>> On Tue, Nov 22, 2016 at 1:44 PM, Rohan Kanade >>>>> wrote: >>>>> >>>>>> Id prefer the first option >>>>>> >>>>>> >>>>>> [Volumes] >>>>>> Volume1.name: tv1 >>>>>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>>>>> Volume1.type: Distribute >>>>>> >>>>>> Volume1.rebalance.files: 0 >>>>>> Volume1.rebalance.data: 0Bytes >>>>>> [Volume1.options] >>>>>> nfs.disable: on >>>>>> performance.readdir-ahead: on >>>>>> transport.address-family: inet >>>>>> features.uss: on >>>>>> >>>>>> Volume2.name: tv2 >>>>>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>>>>> Volume2.type: Distribute >>>>>> ......... >>>>>> ......... >>>>>> >>>>>> >>>>>> This would require minor changes to tendrl/gluster-integration >>>>>> >>>>> definition files and code. I will draw up a spec on >>>> Tendrl/specifications >>>> to document the changes required. Please go ahead with your patch >>>> >>>>> Thanks >>>>>> >>>>>> On Tue, Nov 22, 2016 at 9:18 AM, Atin Mukherjee >>>>>> >>>>> wrote: >>>> >>>>> We are awaiting final confirmation from Rohan. Samikshan has kept the >>>>>>> changes ready and will push it to gerrit once we hear from Rohan. >>>>>>> >>>>>>> On Mon, Nov 21, 2016 at 12:54 PM, Shubhendu Tripathi < >>>>>>> >>>>>> shtripat at redhat.com> >>>> >>>>> wrote: >>>>>>> >>>>>>> Looking at options, I feel option-2 would be more feasible and might >>>>>>>> >>>>>>> not >>>> >>>>> need code changes in tendrl. >>>>>>>> But still lets wait for the confirmation from Rohan. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Shubhendu >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 11/21/2016 12:43 PM, Atin Mukherjee wrote: >>>>>>>> >>>>>>>> +tendrl-devel >>>>>>>>> >>>>>>>>> On Mon, Nov 21, 2016 at 12:41 PM, Samikshan Bairagya < >>>>>>>>> >>>>>>>> sbairagy at redhat.com >>>> >>>>> wrote: >>>>>>>>> >>>>>>>>> Hey Rohan, >>>>>>>>> >>>>>>>>>> So the current get-state CLI misses volume specific options in its >>>>>>>>>> output. >>>>>>>>>> Somehow I missed it while coming up with the implementation. This >>>>>>>>>> >>>>>>>>> patch >>>> >>>>> by >>>>>>>>>> Atin is a fix for that: http://review.gluster.org/#/c/15889/1. The >>>>>>>>>> following example shows how this patch would add these new data >>>>>>>>>> >>>>>>>>> points >>>> >>>>> and >>>>>>>>>> how that would change the existing format: >>>>>>>>>> >>>>>>>>>> [Volumes] >>>>>>>>>> Volume1.name: tv1 >>>>>>>>>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>>>>>>>>> Volume1.type: Distribute >>>>>>>>>> >>>>>>>>>> Volume1.rebalance.files: 0 >>>>>>>>>> Volume1.rebalance.data: 0Bytes >>>>>>>>>> [Volume1.options] >>>>>>>>>> nfs.disable: on >>>>>>>>>> performance.readdir-ahead: on >>>>>>>>>> transport.address-family: inet >>>>>>>>>> features.uss: on >>>>>>>>>> >>>>>>>>>> Volume2.name: tv2 >>>>>>>>>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>>>>>>>>> Volume2.type: Distribute >>>>>>>>>> ......... >>>>>>>>>> ......... >>>>>>>>>> >>>>>>>>>> So essentially there would be a new section for every volume that >>>>>>>>>> >>>>>>>>> would >>>> >>>>> list the option names and corresponding values. Would adding this >>>>>>>>>> >>>>>>>>> change >>>> >>>>> still keep the get-state output parseable from Tendrl POV? >>>>>>>>>> >>>>>>>>>> Or would an output like the following make more sense? Let us know. >>>>>>>>>> Thanks. >>>>>>>>>> >>>>>>>>>> [Volumes] >>>>>>>>>> Volume1.name: tv1 >>>>>>>>>> Volume1.id: 0242f875-24ad-480d-a605-06de2e0f3842 >>>>>>>>>> Volume1.type: Distribute >>>>>>>>>> >>>>>>>>>> Volume1.rebalance.files: 0 >>>>>>>>>> Volume1.rebalance.data: 0Bytes >>>>>>>>>> Volume1.options.nfs.disable: on >>>>>>>>>> Volume1.options.performance.readdir-ahead: on >>>>>>>>>> Volume1.options.transport.address-family: inet >>>>>>>>>> Volume1.options.features.uss: on >>>>>>>>>> >>>>>>>>>> Volume2.name: tv2 >>>>>>>>>> Volume2.id: 937ad30c-bc08-4928-85e4-ece49235037a >>>>>>>>>> Volume2.type: Distribute >>>>>>>>>> ......... >>>>>>>>>> ......... >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> ~ Samikshan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>> Tendrl-devel mailing list >>>>>>>> Tendrl-devel at redhat.com >>>>>>>> https://www.redhat.com/mailman/listinfo/tendrl-devel >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> ~ Atin (atinm) >>>>>>> _______________________________________________ >>>>>>> Tendrl-devel mailing list >>>>>>> Tendrl-devel at redhat.com >>>>>>> https://www.redhat.com/mailman/listinfo/tendrl-devel >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>> Tendrl-devel mailing list >>>> Tendrl-devel at redhat.com >>>> https://www.redhat.com/mailman/listinfo/tendrl-devel >>>> >>> >>> >>> > -------------- next part -------------- [Global] MYUUID: 71eb7804-f922-43e7-b30e-d28a904df497 op-version: 40000 [Global options] cluster.server-quorum-ratio: 60 [Peers] [Volumes] Volume1.name: tv1 Volume1.id: bc06c29b-8ea5-4bd6-9c18-518a9d0b252a Volume1.type: Distribute Volume1.transport_type: tcp Volume1.status: Created Volume1.brickcount: 2 Volume1.Brick1.path: 10.70.1.97:/home/sbairagy/bricks/tb11 Volume1.Brick1.hostname: 10.70.1.97 Volume1.Brick1.port: 0 Volume1.Brick1.rdma_port: 0 Volume1.Brick1.status: Stopped Volume1.Brick1.signedin: False Volume1.Brick2.path: 10.70.1.97:/home/sbairagy/bricks/tb12 Volume1.Brick2.hostname: 10.70.1.97 Volume1.Brick2.port: 0 Volume1.Brick2.rdma_port: 0 Volume1.Brick2.status: Stopped Volume1.Brick2.signedin: False Volume1.snap_count: 0 Volume1.stripe_count: 1 Volume1.replica_count: 1 Volume1.subvol_count: 2 Volume1.arbiter_count: 0 Volume1.disperse_count: 0 Volume1.redundancy_count: 0 Volume1.quorum_status: not_applicable Volume1.snapd_svc.online_status: Offline Volume1.snapd_svc.inited: False Volume1.rebalance.id: 00000000-0000-0000-0000-000000000000 Volume1.rebalance.status: not_started Volume1.rebalance.failures: 0 Volume1.rebalance.skipped: 0 Volume1.rebalance.lookedup: 0 Volume1.rebalance.files: 0 Volume1.rebalance.data: 0Bytes Volume1.options.nfs.disable: on Volume1.options.performance.readdir-ahead: on Volume1.options.transport.address-family: inet Volume1.options.features.uss: on Volume2.name: tv2 Volume2.id: a01c8230-e07a-4bbb-8944-5bb01b14b60e Volume2.type: Distribute Volume2.transport_type: tcp Volume2.status: Created Volume2.brickcount: 2 Volume2.Brick1.path: 10.70.1.97:/home/sbairagy/bricks/tb21 Volume2.Brick1.hostname: 10.70.1.97 Volume2.Brick1.port: 0 Volume2.Brick1.rdma_port: 0 Volume2.Brick1.status: Stopped Volume2.Brick1.signedin: False Volume2.Brick2.path: 10.70.1.97:/home/sbairagy/bricks/tb22 Volume2.Brick2.hostname: 10.70.1.97 Volume2.Brick2.port: 0 Volume2.Brick2.rdma_port: 0 Volume2.Brick2.status: Stopped Volume2.Brick2.signedin: False Volume2.snap_count: 0 Volume2.stripe_count: 1 Volume2.replica_count: 1 Volume2.subvol_count: 2 Volume2.arbiter_count: 0 Volume2.disperse_count: 0 Volume2.redundancy_count: 0 Volume2.quorum_status: not_applicable Volume2.snapd_svc.online_status: Offline Volume2.snapd_svc.inited: False Volume2.rebalance.id: 00000000-0000-0000-0000-000000000000 Volume2.rebalance.status: not_started Volume2.rebalance.failures: 0 Volume2.rebalance.skipped: 0 Volume2.rebalance.lookedup: 0 Volume2.rebalance.files: 0 Volume2.rebalance.data: 0Bytes Volume2.options.nfs.disable: on Volume2.options.performance.readdir-ahead: on Volume2.options.transport.address-family: inet [Services] [Misc] From rkanade at redhat.com Wed Nov 30 13:52:40 2016 From: rkanade at redhat.com (Rohan Kanade) Date: Wed, 30 Nov 2016 19:22:40 +0530 Subject: [Tendrl-devel] Calamari and the Tendrl Ceph Bridge In-Reply-To: References: Message-ID: @gmeno, Could you please point me where to file the agreed RFE's/bugs ? On Tue, Nov 29, 2016 at 5:22 PM, Sharmilla Abhilash wrote: > On Fri, Oct 28, 2016 at 11:06 PM, Gregory Meno wrote: > > > Looks good to me. > > > > On Thu, Oct 20, 2016 at 5:27 AM, Rohan Kanade > wrote: > > > These are the minimum requirements Tendrl has from calamari. If we are > > > agreeing on these, we can file detailed BZ's. > > > > Rohan, all the required BZ's has been filed here ? > > > Gregory, do you you have a time-line when the agreed upon requirements will > be available ? > > > thanks! > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel > From gmeno at redhat.com Wed Nov 30 19:26:17 2016 From: gmeno at redhat.com (Gregory Meno) Date: Wed, 30 Nov 2016 11:26:17 -0800 Subject: [Tendrl-devel] Calamari and the Tendrl Ceph Bridge In-Reply-To: References: Message-ID: Please put them in bugzilla: https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Ceph%20Storage use the component calamari and the version 2.1 On Wed, Nov 30, 2016 at 5:52 AM, Rohan Kanade wrote: > @gmeno, > > Could you please point me where to file the agreed RFE's/bugs ? > > On Tue, Nov 29, 2016 at 5:22 PM, Sharmilla Abhilash > wrote: > >> On Fri, Oct 28, 2016 at 11:06 PM, Gregory Meno wrote: >> >> > Looks good to me. >> > >> > On Thu, Oct 20, 2016 at 5:27 AM, Rohan Kanade >> wrote: >> > > These are the minimum requirements Tendrl has from calamari. If we are >> > > agreeing on these, we can file detailed BZ's. >> > >> >> Rohan, all the required BZ's has been filed here ? >> >> >> Gregory, do you you have a time-line when the agreed upon requirements will >> be available ? >> >> >> thanks! >> _______________________________________________ >> Tendrl-devel mailing list >> Tendrl-devel at redhat.com >> https://www.redhat.com/mailman/listinfo/tendrl-devel >> > _______________________________________________ > Tendrl-devel mailing list > Tendrl-devel at redhat.com > https://www.redhat.com/mailman/listinfo/tendrl-devel From japplewh at redhat.com Wed Nov 30 20:06:16 2016 From: japplewh at redhat.com (Jeff Applewhite) Date: Wed, 30 Nov 2016 15:06:16 -0500 Subject: [Tendrl-devel] Sprint 6 planning meeting Message-ID: Hi All To summarize the sprint planning meeting today: - Most of the priority will be around the "hack days" next week in Bangalore to complete TEN - 156-158. - Another equally high priority is to get to demo (and DoD) for TEN - 43 by way of demonstrating automated functional testing of a Gluster vol create and getting this to "Accepted!" - From there we will pull in or complete these other items: TEN - 43 Create a basic Gluster volume via API. Working to move into accepted. TEN - 84 Performance Monitoring, Alerts, Notifications TEN - 49 SPIKE: Proposal for the monitoring stack in USM TEN - 130 Create APIs for open stack driven creation TEN - 156 Improve Distributed Job orchestration TEN - 157 Refactoring. Waiting for the ?Github ? specs to be completed by COB tomorrow BLR ? - please email list with details? TEN - 158 Generate API documentation automatically TEN - 164 Ceph cluster installation in the UX TEN - 26 Provide remote management access to storage nodes. Should be closed by end of sprint 6. -- Jeff Applewhite Principal Product Manager