[Pulp-dev] pulp 3 PR testing with pulp-smash

[Dennis Kliban]

On Thu, Feb 22, 2018 at 5:21 PM, Daniel Alley <dalley at redhat.com> wrote:

> Would it be possible to have the required tests be Pulp core only, but to
> have an expanded set of non-mandatory smash tests which includes pulp_file?
>
> Which would mean, the pulp_file smash test results would be there as a
> visual indicator, but wouldn't cause problems over the next few months
> before the plugin API is fully stabilized.
>
>
>
Yes we can. We would need to set it up as two separate Jenkins jobs. I
believe GitHub allows setting a subset of checks as gating.



>
> On Thu, Feb 22, 2018 at 4:57 PM, Dennis Kliban <dkliban at redhat.com> wrote:
>
>> tl;dr: which set of pulp-smash tests should run against pulpcore PRs?
>> pulpcore + pulp_file or just pulpcore?
>>
>> Jeremy and I are working to enable a new check for PRs against Pulp's
>> 3.0-dev branch. This is going to be a jenkins job that installs pulpcore
>> from the PR and then runs pulp-smash smoke tests against it.
>>
>> The smoke tests include both pulpcore and pulp_file tests. When testing
>> PRs for pulp repository, should pulp_file also be installed thus allowing
>> pulp-smash all the tests? The other option is to not install pulp_file and
>> allow only the pulpcore tests to run.
>>
>> If both pulpcore and pulp_file tests are required to pass to merge a PR,
>> then we can get into a situation where the plugin API is intentionally
>> changing and the tests can't pass until the change is introduced to
>> pulp_file also. In such situations we could require the pulpcore-plugin
>> package to have it's version bumped, which would mark the build as passing.
>>
>> If only pulpcore tests are run we could get into a situation where the PR
>> breaks the plugin API and we don't learn this until after the code is
>> merged.
>>
>> Which set of tests should run for pulpcore PRs?
>>
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180222/faa3a7d3/attachment.htm>