<div dir="ltr"> <div class="gmail_quote">---------- Forwarded message ---------- From: Robyn Bergeron <<a href="mailto:rbergero@redhat.com">rbergero@redhat.com</a>> Date: Thu, Mar 16, 2017 at 7:30 PM Subject: [OpenStack-Infra] Status of Zuul v3 To: <a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>, <a href="mailto:openstack-infra@lists.openstack.org">openstack-infra@lists.openstack.org</a> Greetings! This periodic update is primarily intended as a way to keep contributors to the OpenStack community apprised of Zuul v3 project status, including future changes and milestones on our way to use in production. Additionally, the numerous existing and future users of Zuul outside of the OpenStack community may find this update useful as a way to track Zuul v3 development status. If "changes are coming in the land of Zuul" is new news to you, please read the section "About Zuul and Zuul v3" towards the end of this email. == Zuul v3 project status and updates == The Big Big news: Updates to Nodepool to support Zuul v3 are done! This has been a large effort (approximately the size of one Stay-Puft marshmallow man), and the team is super excited to now have this in place. Give shrews a high-five if you see him! We still have bugs to shake out, documentation to update, and a backwards-incompatible configuration syntax change to make (<a href="http://lists.openstack.org/pipermail/openstack-infra/2017-January/005018.html" rel="noreferrer" target="_blank">http://lists.openstack.org/pipermail/openstack-infra/2017-January/005018.html</a>), but we can consider it feature complete. Continuing to work on and discuss sample jobs: Putting together an ideal "sample job" and documenting what best practices look like is essential in enabling future users of Zuul to quickly and easily create jobs of their own. Paul Belanger has been making progress on this in the form of a generic tox job (see <a href="https://review.openstack.org/438281" rel="noreferrer" target="_blank">https://review.openstack.org/438281</a>), and we're now at the point of discussing why / how this is a foundational example. Additionally, as more of the major subsystems of Zuul complete their refactoring, the ability for one to contribute to Zuul v3 continues to get a bit easier. Even though at the moment we still recommend having deep familiarity with one or more of the subsystems, we do now have some "low-hanging fruit" tasks listed in storyboard. Use this magical link to find your way: <a href="https://storyboard.openstack.org/#!/story/list?status=active&tags=low-hanging-fruit&project_id=679" rel="noreferrer" target="_blank">https://storyboard.openstack.org/#!/story/list?status=active&tags=low-hanging-fruit&project_id=679</a> Some new specs for enhancements and/or features have been put forth as well: * An interface for Zuul Job Reporting <a href="https://review.openstack.org/444088" rel="noreferrer" target="_blank">https://review.openstack.org/444088</a> * Zuulv3 Executor Security Enhancement <a href="https://review.openstack.org/444495" rel="noreferrer" target="_blank">https://review.openstack.org/444495</a> * Update job trees to graphs <a href="https://review.openstack.org/443985" rel="noreferrer" target="_blank">https://review.openstack.org/443985</a> (yes, technically this is an *update* to a spec, not a new one.) Upcoming tasks and focus: * Re-enabling disabled tests: We're continuing to make our way through the list of remaining tests that need enabling. See the list, which includes an annotation as to complexity for each test, here: <a href="https://etherpad.openstack.org/p/zuulv3skips" rel="noreferrer" target="_blank">https://etherpad.openstack.org/p/zuulv3skips</a> * Full task list and plan is in the Zuul v3 storyboard: <a href="https://storyboard.openstack.org/#!/board/41" rel="noreferrer" target="_blank">https://storyboard.openstack.org/#!/board/41</a> Recent changes: * Zuul v3: <a href="https://review.openstack.org/#/q/status:closed+project:openstack-infra/zuul+branch:feature/zuulv3,25" rel="noreferrer" target="_blank">https://review.openstack.org/#/q/status:closed+project:openstack-infra/zuul+branch:feature/zuulv3,25</a> (Early adoptors should be aware we renamed zuul-launcher to zuul-executor, this is a breaking change: <a href="https://review.openstack.org/#/c/445594" rel="noreferrer" target="_blank">https://review.openstack.org/#/c/445594</a>) * Nodepool: <a href="https://review.openstack.org/#/q/status:closed+project:openstack-infra/nodepool+branch:feature/zuulv3,25" rel="noreferrer" target="_blank">https://review.openstack.org/#/q/status:closed+project:openstack-infra/nodepool+branch:feature/zuulv3,25</a> Previous IRC Meeting minutes & logs: * 2017-03-06 Minutes: <a href="http://eavesdrop.openstack.org/meetings/zuul/2017/zuul.2017-03-06-22.03.html" rel="noreferrer" target="_blank">http://eavesdrop.openstack.org/meetings/zuul/2017/zuul.2017-03-06-22.03.html</a> * 2017-03-06 Full log: <a href="http://eavesdrop.openstack.org/meetings/zuul/2017/zuul.2017-03-06-22.03.log.html" rel="noreferrer" target="_blank">http://eavesdrop.openstack.org/meetings/zuul/2017/zuul.2017-03-06-22.03.log.html</a> * 2017-03-13 Minutes: <a href="http://eavesdrop.openstack.org/meetings/zuul/2017/zuul.2017-03-13-22.02.html" rel="noreferrer" target="_blank">http://eavesdrop.openstack.org/meetings/zuul/2017/zuul.2017-03-13-22.02.html</a> * 2017-03-13 Full log: <a href="http://eavesdrop.openstack.org/meetings/zuul/2017/zuul.2017-03-13-22.02.log.html" rel="noreferrer" target="_blank">http://eavesdrop.openstack.org/meetings/zuul/2017/zuul.2017-03-13-22.02.log.html</a> == About Zuul and Zuul v3 == Zuul is a pipeline-oriented project gating system, driving the project automation necessary to enable a continuous integration environment for the OpenStack community. It directs the OpenStack community’s testing, running tens of thousands of jobs each day, responding to events from the code review system and stacking potential changes to be tested together. Testing and continuous integration for drivers is also enabled by OpenStack’s deployment of Zuul for ~50 third-party projects. Zuul v3 is the upcoming, not-yet-in-production version of Zuul currently under development, bringing (amongst many others) the following improvements and features: * simplifying the ability to run jobs in multi-node environments * improved management of large numbers of jobs and job variations * support for in-tree job configuration * ability to define jobs using Ansible (<a href="http://github.com/ansible/ansible" rel="noreferrer" target="_blank">http://github.com/ansible/ansible</a>) Even though prior versions of Zuul are already in use by numerous projects and companies not related to OpenStack efforts, a primary goal of Zuul v3 is to make Zuul a universally useful CI / CD engine for any size use case. The design of Zuul v3 improves the modularity of the system, and enables new triggers (such as GitHub) and reporters (where results are sent) to be more easily integrated with Zuul; additionally, the ability to execute jobs on non-OpenStack clouds, such as, well, the other ones :D, as well as non-cloud environments, will be possible. (These "beyond OpenStack" changes are expected in Zuul 3.1, which will follow the moment after Zuul v3 is live and in production for the OpenStack community. Finally, the move from Jenkins to Ansible, in addition to improving scalability for large implementations, allows users to write tests using Ansible, effectively allowing one to “test in the way that you deploy,” and enables simpler writing of test jobs. OpenStack’s current production use of Zuul supports approximately 2500 developers, more than 1500 git repositories, at a rate of more than 2KJPH (kilo-jobs / hour). For more background and information on the history, and future, of Zuul, please see: * Zuul v3 spec: <a href="https://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html" rel="noreferrer" target="_blank">https://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html</a> * Jim Blair’s Zuul v3 talk at OpenStack Summit in Barcelona: <a href="https://www.youtube.com/watch?v=R4EmE1QEvNU" rel="noreferrer" target="_blank">https://www.youtube.com/watch?v=R4EmE1QEvNU</a> * “There is no Jenkins, only Zuul” (blog post, Jim Blair, 2016/06/16): <a href="http://amo-probos.org/post/18" rel="noreferrer" target="_blank">http://amo-probos.org/post/18</a> * "Zuul v3, What’s Coming" (blog post, Monty Taylor, 2017/02/28): <a href="http://inaugust.com/posts/whats-coming-zuulv3.html" rel="noreferrer" target="_blank">http://inaugust.com/posts/whats-coming-zuulv3.html</a> Significant dates, milestones, and changes directly affecting contributors in the OpenStack community will be announced on the appropriate mailing lists for OpenStack (primarily openstack-dev and openstack-infra), as well as in future versions of this project status update. NOTE: At this point, Zuul v3 is NOT yet secure or stable. You should not, not, please, DO NOT ATTEMPT TO RUN IT IN PRODUCTION YOURSELF yet. It is still under heavy development, and v3 is not currently used in production for OpenStack. Zuul v3 content (jobs, etc.) should not undergo heavy creation as syntax is still under development and expected to change between now and release date. == Zuul Community Information == Zuul v3 Storyboard, which tracks tasks and status related to operating Zuul v3 in production for OpenStack: <a href="https://storyboard.openstack.org/#!/board/41" rel="noreferrer" target="_blank">https://storyboard.openstack.org/#!/board/41</a> Mailing list: Discussions related to Zuul development, and current / future operation in production in the OpenStack community, primarily occur on openstack-infra (<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra</a>). (Note that this mailing list encompasses the numerous infrastructure projects supported by OpenStack’s infrastructure team, and is not exclusively Zuul-focused.) IRC Discussion: Join #openstack-infra on Freenode to discuss Zuul's use in OpenStack or #zuul to get into the weeds on general Zuul topics and v3 development. For those unacquainted with IRC, see this friendly guide to using IRC: <a href="https://wiki.openstack.org/wiki/IRC" rel="noreferrer" target="_blank">https://wiki.openstack.org/wiki/IRC</a> Zuul v3 Weekly IRC meeting: * Mondays at 2200UTC, in #openstack-meeting-alt on Freenode * Historical meeting logs: <a href="http://eavesdrop.openstack.org/meetings/zuul/" rel="noreferrer" target="_blank">http://eavesdrop.openstack.org/meetings/zuul/</a> * Further information, including upcoming meeting agenda: <a href="https://wiki.openstack.org/wiki/Meetings/Zuul" rel="noreferrer" target="_blank">https://wiki.openstack.org/wiki/Meetings/Zuul</a> Cheers, -Robyn (on behalf of the openstack-infra team) _______________________________________________ OpenStack-Infra mailing list <a href="mailto:OpenStack-Infra@lists.openstack.org">OpenStack-Infra@lists.openstack.org</a> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra</a></div> </div>