[Avocado-devel] Post 82.0 LTS plans

[Beraldo Leal]

On Wed, Sep 16, 2020 at 03:02:58PM -0400, Cleber Rosa wrote:
> Hi Avocado community,
> 
> 69.x LTS
> ========
> 
> The general rule that the previous LTS version will be maintained for
> a minimum of 6 months following another LTS release.  Following that
> rule, we would be maintaining the 69.x LTS series until March 2021.
> 
> One important characteristic of the 69.x series is that it supports
> Python 2, which has been declared EOL on Dec 2019.  So users stuck
> with Python 2 can only use 69.x LTS.  In theory, the March 2021 date
> far exceeds the Python 2 EOL date, so we suppose that it's a safe
> date.

+1 here.

> 
> Avocado-VT and Avocado Compatibility
> ====================================
> 
> Avocado-VT is Avocado's primary "customer" and the two share a common
> origin, and one retro feeds improvements into the other.  Having said
> that, a lot of energy has been spent into keeping Avocado-VT compatible
> with multiple Avocado of versions.  At any given moment, Avocado-VT is
> supposed to be compatible with:
> 
> 1. Avocado 69.x LTS
> 2. Avocado Latest release (currently 82.0)
> 3. Avocado development branch
> 
> With Avocado 82.0 LTS, there's an opportunity to improve this
> situation.  The first drastic change I'll propose is to drop as many
> of the compatibility requirements above, and pin Avocado-VT for the
> time being to require compatibility Avocado 82.0 LTS *only*.  This
> would greatly simplify Avocado-VT's development, as there would be
> one and only way to deal with Avocado.
> 
> On the other hand, this could cause:
> 
> 1. Bitrot, where Avocado-VT would at a given point not work with
>    newer Avocado, say the next LTS version.
> 
> 2. A "lock down", where improvements to "avocado.utils" libraries
>    would not be easily available to Avocado-VT tests.
> 
> Avoiding bitrot
> ---------------
> 
> Let's assume that the next LTS release is version 95.0.  The worst
> possible scenario would be "waking up" and finding out that making
> Avocado-VT compatible Avocado 95.0 would be overwhelming and
> prohibitive mission.
> 
> One mitigation would be to extend 82.0 LTS life time, but this would
> only push the problem further away and not really solve it.
> 
> To avoid such a situation, I propose that a proactive effort is
> made every few Avocado releases, with the goal of:
> 
> * evaluate the current compatibility
> * plan and develop compatibility tasks
> 
> Those should be done not only in light of the *current* Avocado state,
> but also in sync with Avocado's planned features.  For example,
> consider the following release time line:
> 
> * Avocado 83.0 released
> * Avocado 84.0 released
> * Avocado 85.0 released
> * Avocado-VT compatibility evaluated with Avocado 85.0
>  - Compatibility with newer Avocado "runner API" identified and
>    issue created
>  - Compatibility with newer Avocado "result API" identified and
>    issue created
> * Avocado 86.0 is released
>  - Compatibility with "runner API" resolved
> * Avocado 87.0 is released
>  - Compatibility with "result API" resolved
> * Avocado 88.0 released
> * Avocado-VT compatibility evaluated with Avocado 88.0
>   ...
> * Avocado 94.0 released (pre LTS release)
>  - All pending compatibility issues are addressed
> * Avocado 95.0 released
>  - Avocado-VT should be compatible with both current 95.0 LTS and 82.x
>    LTS
> 
> Integration jobs on CI can also keep running with Avocado master, but
> in non-gating state ("allow failures" in Travis CI terminology).  This
> can serve as a "temperature check" for the level of compatibility and
> tasks to records as issues and address in the near future.

My only concern here is having too many issues tagged as "lts
compatibility" and we need to fix all those issues in the last minute
right before the avocado-vt release.

But I agree with you: 1. If Avocado-vt it is running in critical systems
would be better to not use avocado's master; 2. This proposal will give
us more time to fix those issues until next LTS.

We just need to get alerts on CI, like you said, to make sure that we
are aware of those breaks.

> 
> Library lock downs
> ------------------
> 
> Users running tests can usually wait a reasonable long time for new
> features to land in the Avocado test runner.  In contrast, fixes or
> new features in utility libraries are usually needed to write a test
> and can not wait for too long.
> 
> With the proposal above to bind Avocado-VT to Avocado LTS versions
> only, utility libraries present within Avocado that is,
> "avocado.utils", would also be limited to the LTS version.  This is
> not desirable, as it can:
> 
> * Slow down test development
> 
> * Create duplicate copies of utilities:
>  - within different tests
>  - across Avocado and Avocado-VT
> 
> Moving all utility libraries into Avocado-VT (such as the one currently
> available in Avocado) is also not an option, as users of other tests
> (for example avocado-misc-tests) and even Avocado itself need them.
> 
> The most obvious solution is to move the utility libraries to its own
> repository.  Let's suppose this new repository/module is called
> "greenmatter" (my imagination ran wild, please do the same ;).  This
> would allow for scenarios such as:
> 
> 1. a) Avocado-VT master
>    b) Avocado LTS (with its builtin current version of "avocado.utils")
>    c) new tp-qemu tests using newer "greenmatter" utility libraries
> 
> 2. a) avocado-misc-tests master
>    b) Avocado (either LTS or latest)
>    c) new tests using newer "greenmatter" utility libraries

+10 for moving utilities for a common repository. Today we are spending
many hours and maintaining multiple libraries in both places.

We could concentrate the efforts in one place. We just need to ensure a
good code quality review to avoid chaos. Maybe a blueprint for
describing the requirements for a new library, could be useful here.

> Goals of "greenmatter" libraries
> --------------------------------
> 
> Common API "look and feel"
> ~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> Getting the design of APIs right is pretty hard, but here we have a
> unique change of declaring a common set of guidelines from the start
> and enforcing as libraries are added.
> 
> It's expected that a "BluePrint" would be written defining, among other
> things:
> 
>  * how modules should be named
>  * how classes should be named
>  * how functions should be named if they return information
>  * how functions should be named if they check or assert some condition
>  * how functions should behave if running on a platform that doesn't
>    apply (say "distro" is found running on a BSD or Windows system)

I guess you answered my previous comment here.

Just one additional note: I'm currently drafting a BP000 with details
about how to present a blueprint and possible blueprint types. I will
send a RFC as soon as possible.

--
Beraldo