[Avocado-devel] [RFC v3]: Avocado maintainability and integration with avocado-vt

Cleber Rosa crosa at redhat.com
Wed Apr 26 23:01:37 UTC 2017



On 04/25/2017 12:05 PM, Lukáš Doktor wrote:
> Dne 20.4.2017 v 03:42 Cleber Rosa napsal(a):
>> Hi Folks.
>>
> Hello Cleber,
> 
> thank you for the updates, in general it's good, I have few minor
> suggestions in-line.
> 
>> This RFC contains proposals and clarifications regarding the
>> maintenance and release processes of Avocado.
>>
>> We understand there are multiple teams currently depending on the
>> stability of Avocado and we don't want their work to be disrupted by
>> incompatibilities nor instabilities in new releases.
>>
>> This version is a minor update to RFC version 2[1], which drove the
>> release of Avocado 36.0 LTS.  The Avocado team has plans for a new LTS
>> release in the near future, so please consider reading and providing
>> feedback on the proposals here.
>>
>> TL;DR:
>>
>>   We plan to keep the current approach of sprint releases every 3-4
>>   weeks, but we're introducing "Long Term Stability" releases which
>>   should be adopted in production environments where users can't keep
>>   up with frequent upgrades.
>>
>> Changes from v2:
>>   - Wording changes on second paragraph ("... nor instabilities...")
>>   - Clarified on "Introduction" that change of behavior is introduced
>>     between regular releases
>>   - Updated distro versions for which official packages are built
>>   - Add more clear explanation on official packages on the various
>>     hardware platforms
>>   - Used more recent version numbers as examples, and the planned
>>     new LTS version too
>>   - Explain how users can get the LTS version when using tools such as
>>     pip
>>   - Simplified the timeline example, with examples that will possibly
>>     match the future versions and releases
>>   - Documented current status of avocado-vt releases and packages
>>
>> Changes from v1:
>>   - Changed "Support" to "Stability" and "supported" to "maintained"
>>     [Jeff Nelson]
>>   - Misc improvements and clarifications in the
>>     supportability/stability statements [Jeff Nelson, me]
>>   - Fixed a few typos [Jeff Nelson, me]
>>
>>
>> Introduction
>> --------------
>>
>> We make new releases of Avocado every 3-4 weeks on average.  In theory
>> at least, we're very careful with backwards compatibility.  We test
>> Avocado for regressions and we try to document any issues, so
>> upgrading to a new version should be (again, in theory) safe.
>>
>> But in practice both intended and unintended changes are introduced
>> during development, and both can be frustrating for conservative
>> users. We also understand it's not feasible for users to upgrade
>> Avocado very frequently in a production environment.
>>
>> The objective of this RFC is to clarify our maintenance practices and
>> introduce Long Term Stability (LTS) releases, which are intended to
>> solve, or at least mitigate, these problems.
>>
>>
>> Our definition of maintained, or stable
>> ---------------------------------------
>>
>> First of all, Avocado and its sub-projects are provided 'AS IS' and
>> WITHOUT ANY WARRANTY, as described in the LICENSE file.
>>
>> The process described here doesn't imply any commitments or
>> promises. It's just a set of best practices and recommendations.
>>
>> When something is identified as "stable" or "maintained", it means the
>> development community makes a conscious effort to keep it working and
>> consider reports of bugs and issues as high priorities.  Fixes
>> submitted for these issues will also be considered high priorities,
>> although they will be accepted only if they pass the general
>> acceptance criteria for new contributions (design, quality,
>> documentation, testing, etc), at the development team discretion.
>>
>>
>> Maintained projects and platforms
>> ---------------------------------
>>
>> The only maintained project as of today is the Avocado Test Runner,
>> including its APIs and core plugins (the contents of the main avocado
>> git repository).
>>
>> Other projects kept under the "Avocado Umbrella" in github may be
>> maintained by different teams (e.g.: avocado-vt) or be considered
>> experimental (e.g.: avocado-server and avocado-virt).
>>
>> More about avocado-vt in its own section further down.
>>
>> As a general rule, fixes and bug reports for Avocado when running in
>> any modern Linux distribution are welcome.
>>
>> But given the limited capacity of the development team, packaged
>> versions of Avocado will be tested and maintained only for the
>> following Linux distributions:
>>
>>   * RHEL 7.x (latest)
>>   * Fedora (stable releases from the Fedora projects)
>>
>> Currently all packages produced by the Avocado projects are "noarch".
>> That means that they could be installable on any hardware platform.
>> Still, the development team will currently attempt to provide versions
>> that are stable for the following platforms:
>>
>>   * x86
>>   * ppc64
> Do we still care about ppc64 BE? And how about the widely spreading
> aarch64, do we want to include it?
> 

Honestly, I can't properly answer that.

Jeff, can you help us here?

>>   * ppc64le
>>
>> Contributions from the community to maintain other platforms and
>> operating systems are very welcome.
>>
>> The lists above may change without prior notice.
>>
>>
>> Avocado Releases
>> ----------------
>>
>> The proposal is to have two different types of Avocado releases:
>>
>> 1. Sprint Releases:
>>   (This is the model we currently adopt in Avocado)
>>
>>   They happen every 3-4 weeks (the schedule is not fixed) and
>>   their versions are numbered serially, with decimal digits in
>>   the format <major>.<minor>.  Examples: 47.0, 48.0, 49.0.  Minor
>>   releases are rare, but necessary to correct some major issue
>>   with the original release (47.1, 47.2, etc).
>>
>>   Only the latest Sprint Release is maintained.
>>
>>   In Sprint Releases we make a conscious effort to keep backwards
>>   compatibility with the previous version (APIs and behavior) and
>>   as a general rule and best practice, incompatible changes in
>>   Sprint Releases should be documented in the release notes and
>>   if possible deprecated slowly, to give users time to adapt
>>   their environments.
>>
>>   But we understand changes are inevitable as the software
>>   evolves and therefore there's no absolute promise for API and
>>   behavioral stability.
>>
>> 2. Long Term Stability (LTS) Releases:
>>
>>   LTS releases should happen whenever the team feels the code is
>>   stable enough to be maintained for a longer period of time, ideally
>>   once or twice per year (no fixed schedule).
>>
>>   They should be maintained for 18 months, receiving fixes for major
>>   bugs in the form of minor (sub-)releases. With the exception of
>>   these fixes, no API or behavior should change in a minor LTS
>>   release.
>>
>>   They will be versioned just like Sprint Releases, so looking at the
>>   version number alone will not reveal the differentiate release
>>   process and stability characteristics.
>>
>>   In practice each major LTS release will imply in the creation of a
>>   git branch where only serious issues affecting users will be
>>   fixed. The code in a LTS branch is stable, frozen for new features.
>>
>>   Notice that although within a LTS release there's a expectation
>>   of stability because the code is frozen, different (major) LTS
>>   releases may include changes in behavior, API incompatibilities
>>   and new features. The development team will make a considerable
>>   effort to minimize and properly document these changes (changes
>>   when comparing it to the last major LTS release).
>>
>>   Sprint Releases are replaced by LTS releases. I.e., in the cycle
>>   when 52.0 (LTS) is released, that's also the version used as a
>>   Sprint Release (there's no 52.0 -- non LTS -- in this case).
>>
>>   New LTS releases should be done carefully, with ample time for
>>   announcements, testing and documentation.  It's recommended
>>   that one or two sprints are dedicated as preparations for a LTS
>>   release, with a Sprint Release serving as a "LTS beta" release.
>>
>>   Similarly, there should be announcements about the end-of-life
>>   (EOL) of a LTS release once it approaches its 18 months of
>>   life.
>>
>>
>> Misc details
> How about calling this section deployment details?
> 

Good idea.

>> ------------
>>
>> Sprint and LTS releases, when packaged, whenever possible, will be
>> preferably distributed through different package channels
>> (repositories).
>>
>> This is possible for repository types such as YUM/DNF repos[2].  In
>> such cases, users can disable the regular channel, and enable the LTS
>> version.  A request for the installation of Avocado packages will
>> fetch the latest version available in the enabled repository.  If the
>> LTS repository channel is enabled, the packages will receive minor
>> updates (bugfixes only), until a new LTS version is released (roughly
>> every 12 months).
>>
>> If the non-LTS channel is enabled, users will receive updates every
>> 3-4 weeks.
>>
>> On other types of repos such as PyPI[3] which have no concept of
>> "sub-repos" or "channels", users can request a version smaller than
>> the version that succeeds the current LTS to get the latest LTS
>> (including minor releases).  Suppose the current LTS major version is
>> 52, but there have been minor releases 52.1 and 52.2.  By running:
>>
>>    $ pip install 'avocado-framework<53.0'
>>
>> pip provide LTS version 52.2.  If 52.3 gets released, they will be
>> automatically deployed instead.  When a new LTS is released, users
>> would still get the latest minor release from the 52.0 series, unless
>> they update the version specification.
>>
>> The existence of LTS releases should never be used as an excuse
>> to break a Sprint Release or to introduce gratuitous
>> incompatibilities there. In other words, Sprint Releases should
>> still be taken seriously, just as they are today.
>>
>>
>> Timeline example
>> ----------------
>>
>> Assume each sprint is taking 3 weeks.  Notice how multiple LTS
>> releases can co-exist before EOL.
>>
> It should start with:
> 
>     LTS release 36.0 (previous LTS release)
>     ...
> 

Good point.

>>     sprint release 49.0
>>     sprint release 50.0
>>        --> start preparing a LTS release, so 51.0 is a beta LTS
>>     sprint release 51.0
>>     LTS release 52.0 (52lts branch is created, packages go into LTS repo)
>>     sprint release 53.0
>>       --> bug that also affects 52.0 is found, fix gets added to
>>           master and 52lts branches
>>     sprint release 54.0 + LTS 52.1
>>       --> bug that also affects 52.0 is found, fix gets added to
> it affects 52.1
> 
> Also I'd mention that it was a minor bug (as we did not bother releasing
> a new version not even when new sprint was released.
> 

Yep, good point.

Thanks for the feedback,
- Cleber.

-- 
Cleber Rosa
[ Sr Software Engineer - Virtualization Team - Red Hat ]
[ Avocado Test Framework - avocado-framework.github.io ]
[  7ABB 96EB 8B46 B94D 5E0F  E9BB 657E 8D33 A5F2 09F3  ]

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://listman.redhat.com/archives/avocado-devel/attachments/20170426/5b7cbaa5/attachment.sig>


More information about the Avocado-devel mailing list