[Pulp-dev] Proposal: Basic Functional Test Requirement with each Feature or Bugfix

Pavel Picka ppicka at redhat.com
Tue Aug 18 08:37:38 UTC 2020

Big +1 to require (at least) one basic/positive functional test if
Maybe we can set up a review checklist (contains 'check for basic test').

We already have some docs which we can update a bit [0] to be yet more
plugin writer friendly and point back to it from plugins' docs.
Like extending it with the way the test can be run when pulp is installed
e.g. by the rpm package.

[0] https://docs.pulpproject.org/contributing/tests.html

On Mon, Aug 17, 2020 at 10:07 PM Brian Bouterse <bmbouter at redhat.com> wrote:

> I have two goals with this:
> 1. Improve the stability of pulpcore and it's plugins
> 2. Enable all downstreams and packagers with tests and information on how
> to run those tests, so they can run the automated tests even as the
> downstream is curated with a series of backports. This closely follows the
> model of a downstream maintainer who backports a feature/fix from upstream
> into a RPM and runs the upstream tests to make sure their patch didn't
> break functionality.
> # Proposal
> Upstream could require a basic functional test for each feature or a test
> for each bug fix. This test would be in the same commit as the feature or
> fix causing it to "follow the code" so when it's cherry picked downstream
> or into a package, the test goes with it. I believe we don't change the
> test during cherry pick time because although the code implementation may
> change what the feature or fix provides will not.
> Additionally, upstream should document how to run these tests easily and
> have the goal of making that easy.
> This policy change would ultimately be decided by each plugin and pulpcore
> separately. I don't want to make it a requirement to be part of the
> github.com/pulp/ organization in any way, so if a plugin can't, or won't,
> make this change that's ok, they can still host at github.com/pulp/.
> As a pulpcore and pulp_file maintainer, I'm proposing this policy for both
> of those repos specifically.
> In terms of enforcement, I'm pitching manual enforcement by review as a
> start. We can automate it later, but I hope that's a separate discussion to
> focus on the policy change and keep it simple.
> # Brian's take
> This is a non-trivial contribution requirement so we need to really think
> about it. My personal take is that at this point in the 3.y release line
> it's time to trade feature/fix velocity for stability. Also backporting is
> about to become a regular activity, so I think we have a practical
> motivation to adopt this, even though it will slow down the future features
> and bug fix velocity.
> What's your perspective?
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev

Pavel Picka
Red Hat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20200818/ff0eaf77/attachment.htm>

More information about the Pulp-dev mailing list