[Pulp-dev] RFC: Y Release Cadence Proposal

[Sean Myers]

I think the current Y Release Cadence has proven to be untenable.

Here's a summary of what I think about it based on our experience:
* Branching at the first build (beta 1 release) works very well, so we should keep doing that.
* 6-week GA-to-GA dev windows are too small, so we should increase this to 9 weeks (maybe more)
* Our betas seem to always catch a few blockers, so the timeline always slips between beta and RC,
  and might slip again between RC and GA. This is completely normal, and is in fact why we have
  Beta and RC stages, so this uncertainty should be built into the process.
* I don't know that we ever really respected the feature freeze; it seemed to me like the knowledge
  that we were cutting a beta in a week was "good enough", and there's no need to formally freeze
  development. Our three-week sprint cycles also do a good job of effectively feature-freezing
  every three weeks.
* "Resetting the clock" after each beta, and especially after each release candidate, doesn't
  seem to add stability. Instead, it seems to add frustration in the form of release delays. The
  decision to advance from Beta to RC and RC to GA should be handled per-release, with the only
  strictly enforced time restriction being that a minimum of one week should pass before advancing
  to the next phase is allowed.

With those points in mind, the proposed changes are to release the first beta in a time-based way,
9 weeks after the release date of the first beta of the previous Y release. Once a beta is released,
no release timelines are guaranteed for the progression from beta to RC to GA beyond reasonable
minimums (currently 1 week minimum testing time for a beta, and 1 week minimum testing time for RC).

Scheduling from beta to beta isolates the release process from the normal pre-release rebuilding
delays that happen as a result of fixing blocking issues, while still giving the release schedule
predictability so folks can do reasonable planning around our release dates.

With all that preamble, here's my proposed "Y" release schedule:

---

The Y release cycle begins on the day of the previous *Beta* Y release.

* week 0: (Y-1 Release) Previous Y release cycle begins with a Beta Release
* week 9: (Y Release) Y release cycle begins with a Beta Release
** dev branch created for this Y Release
** build system adapted to build from dev branch
** master branch now tracks development for next Y release
** beta is built from dev branch
* week 9+1?: Y release advances through Beta phase to RC as phase conditions are met, with
            a minimum time in Beta of one week. Release schedule is no longer time-based.
* week 9+2?: Y release advances through RC phase to GA as phase conditions are met, with
            a minimum time in RC of one week.

---

The phase conditions mentioned in this schedule are listed on the linked wiki page already,
but should be more clearly defined. Along with this proposed schedule, here are my proposed
definitions for those terms:

Beta: All bug fixes that need to be in the release are complete. Dev believes the release is ready.
      For Y releases, verification of the new functionality begins.
Release Candidate (RC): Verification of new functionality is completed. No new bug fixes accepted
                        after this point except fixes for regressions, upgrade failures, or security.
                        There are no release candidates for Z releases.
Generally Available (GA): Unchanged.
Dev branch: Unchanged.
Feature Freeze: This definition should go away if feature freeze is removed from our "Y" release schedule.
                If we keep the feature freeze step in, this definition can remain unchanged.

This schedule is less structured that the previous iteration, but I think it more accurately
reflects how our releases actually go.

The 2.10.0 beta was first released August 4, which would mean that under this schedule we would
have begun the 2.11 release process last week. That feels about right to me, if we were staying
on a strict time-based release cadence for 2.y, but going back to a 12-week (quarterly-ish)
schedule would probably also be reasonable.

For reference, here's the current "Y" release cadence posted in the project wiki[0]:

---

The Y release cycle begins on the day of the previous GA Y release.

* week 0: (Y-1) Previous Y Generally Available Release
* week 3: Feature Freeze
* week 4: Beta Y release
** dev branch created for this Y Release
** build system adapted to build from dev branch
** master branch now tracks development for next Y release
** beta is built from dev branch
* week 5: Y Release Candidate
* week 6: Generally Available Y Release

---

More words here because this email doesn't have enough words...

[0]: https://pulp.plan.io/projects/pulp/wiki/Release_Schedule

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20161014/d646dd97/attachment.sig>