[Pulp-dev] Proposal: Basic Functional Test Requirement with each Feature or Bugfix
fabricio.aguiar at redhat.com
Tue Aug 18 14:12:14 UTC 2020
I think we can do silly automation in one specific plugin, after some
trial, expand the idea or not:
pulpcore on 6844 [$] via 🐍 v3.8.0 (venv)
❯ git diff --name-only master
pulpcore on 6844 [$] via 🐍 v3.8.0 (venv) took 2s
❯ git diff --name-only master | grep -E 'feature|bugfix'
pulpcore on 6844 [$] via 🐍 v3.8.0 (venv)
❯ git diff --name-only master | grep -E '^test_'
Software Engineer, Pulp Project
Red Hat Brazil - Latam <https://www.redhat.com/>
+55 11 999652368
On Tue, Aug 18, 2020 at 11:08 AM David Davis <daviddavis at redhat.com> wrote:
> +1 from me.
> One of the problems I foresee though is that this could make cherry
> picking difficult if we have tests that depend on other tests. For example,
> suppose you have change A that adds some tests and then change B adds some
> tests on top of A's tests. It'll make cherry picking B without A tricky.
> We'll need to be careful to avoid such situations.
> On Tue, Aug 18, 2020 at 4:48 AM Matthias Dellweg <mdellweg at redhat.com>
>> On Tue, Aug 18, 2020 at 10:39 AM Pavel Picka <ppicka at redhat.com> wrote:
>> > Big +1 to require (at least) one basic/positive functional test if
>> > Maybe we can set up a review checklist (contains 'check for basic
>> > We already have some docs which we can update a bit  to be yet more
>> plugin writer friendly and point back to it from plugins' docs.
>> > Like extending it with the way the test can be run when pulp is
>> installed e.g. by the rpm package.
>> >  https://docs.pulpproject.org/contributing/tests.html
>> > On Mon, Aug 17, 2020 at 10:07 PM Brian Bouterse <bmbouter at redhat.com>
>> >> I have two goals with this:
>> >> 1. Improve the stability of pulpcore and it's plugins
>> >> 2. Enable all downstreams and packagers with tests and information on
>> how to run those tests, so they can run the automated tests even as the
>> downstream is curated with a series of backports. This closely follows the
>> model of a downstream maintainer who backports a feature/fix from upstream
>> into a RPM and runs the upstream tests to make sure their patch didn't
>> break functionality.
>> >> # Proposal
>> >> Upstream could require a basic functional test for each feature or a
>> test for each bug fix. This test would be in the same commit as the feature
>> or fix causing it to "follow the code" so when it's cherry picked
>> downstream or into a package, the test goes with it. I believe we don't
>> change the test during cherry pick time because although the code
>> implementation may change what the feature or fix provides will not.
>> >> Additionally, upstream should document how to run these tests easily
>> and have the goal of making that easy.
>> >> This policy change would ultimately be decided by each plugin and
>> pulpcore separately. I don't want to make it a requirement to be part of
>> the github.com/pulp/ organization in any way, so if a plugin can't, or
>> won't, make this change that's ok, they can still host at
>> >> As a pulpcore and pulp_file maintainer, I'm proposing this policy for
>> both of those repos specifically.
>> >> In terms of enforcement, I'm pitching manual enforcement by review as
>> a start. We can automate it later, but I hope that's a separate discussion
>> to focus on the policy change and keep it simple.
>> >> # Brian's take
>> >> This is a non-trivial contribution requirement so we need to really
>> think about it. My personal take is that at this point in the 3.y release
>> line it's time to trade feature/fix velocity for stability. Also
>> backporting is about to become a regular activity, so I think we have a
>> practical motivation to adopt this, even though it will slow down the
>> future features and bug fix velocity.
>> >> What's your perspective?
>> >> _______________________________________________
>> >> Pulp-dev mailing list
>> >> Pulp-dev at redhat.com
>> >> https://www.redhat.com/mailman/listinfo/pulp-dev
>> > --
>> > Pavel Picka
>> > Red Hat
>> > _______________________________________________
>> > Pulp-dev mailing list
>> > Pulp-dev at redhat.com
>> > https://www.redhat.com/mailman/listinfo/pulp-dev
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
> Pulp-dev mailing list
> Pulp-dev at redhat.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Pulp-dev