[Pulp-dev] Pull Request builder job changes for plugins

[Brian Bouterse]

@dkliban, I think that is a good solution. Since it's currently using the
latest GA version of core, all we need to do is extend the current job as
you describe. I rewrite the issue [0] to reflect that last part of the
work. This is on the current sprint at NEW so I imagine it will get picked
up soon. If we should do something different feel free to rewrite.

[0]: https://pulp.plan.io/issues/2751

-Brian

On Mon, Jun 5, 2017 at 10:45 AM, Dennis Kliban <dkliban at redhat.com> wrote:

> What if we ran our plugin unit tests against both the latest GA build and
> nightly build of core?
>
> If the tests pass with the GA version, the job is marked as successful. If
> not, core packages are upgraded to the latest nightly and unit tests are
> run again. If the unit tests fail again, the job is marked as failed. If
> the unit tests pass with the latest nightly, the job is marked as
> successful.
>
> On Sun, Jun 4, 2017 at 12:03 PM, Brian Bouterse <bbouters at redhat.com>
> wrote:
>
>> After thinking about this more, I realized that for the remainder of
>> Pulp2 at least, we need to have the plugin unittest runner test against the
>> nightly version of core and not the latest GA. Using the GAs won't work
>> because not only is the 'I need unreleased code from platform' a problem
>> with the PR that needs it but also a problem for all subsequent PRs after
>> its merged. That second part makes using GA core as the basis for plugin
>> testing probably a non-starter. Assuming that, the next step for 2751 is to
>> update the GA urls to be the "stable nightly" URLs.
>>
>> We also need to look into the nightlies to check on their reliability. In
>> theory, each night an "unstable nightly" of core gets built nightly in
>> Jenkins, tested with pulp-smash, and if all tests pass it gets "promoted"
>> to a separate URL for "stable nightlies". Let me know if we should move
>> this to another thread, but I've got these questions about nightlies.
>>
>> 1. Who investigates when the "unstable nightly" fails to build?
>> 2. Who investigates when a "unstable nightly" fails to be promoted to a
>> "stable nightly" due to pulp-smash failures?
>> 3. Who is in charge of maintaining these Jenkins jobs over time and are
>> they currently maintained?
>> 4. Who is in charge of managing the directory structure on
>> repos.fedorapeople.org?
>> 5. Where are the docs on ^?
>>
>> With Pulp3 I think we can switch to using the latest GA as the basis for
>> plugin testing which would be better in several ways.
>>
>> -Brian
>>
>> On Fri, Jun 2, 2017 at 5:06 PM, Brian Bouterse <bbouters at redhat.com>
>> wrote:
>>
>>> That is a good point, and one we are giving some thought to through
>>> convo on #pulp-dev and the issue [0]. The case of a plugin needing an
>>> unreleased change from core would fail with this change. It's a tradeoff
>>> though because if we go with nightlies as the version of core that is used,
>>> whenever the nightlies break, the unittest PR runners also will, which has
>>> been a reliability issue with the plugin unittest runner for a while.
>>>
>>> I wrote some on the issue about it, but I see the 'plugin needs
>>> unreleased code from core' as a special case, not a normal case. It used to
>>> be common, but it's getting less common, which is good, because
>>> contributing to a plugin should not involve changes to the core as the
>>> norm. It will happen from time to time, so we can handle the special case,
>>> specially by running the unittests locally with the necessary unreleased
>>> version of platform and posting the results as evidence that its safe to
>>> merge.
>>>
>>> [0]: https://pulp.plan.io/issues/2751
>>>
>>> On Fri, Jun 2, 2017 at 4:43 PM, Michael Hrivnak <mhrivnak at redhat.com>
>>> wrote:
>>>
>>>> What about cases where a plugin wants to use something that's new in
>>>> the unreleased core? The master branch of a plugin will usually be released
>>>> with the master branch of the core in the next 2.y release for example.
>>>> That seems like a normal scenario; is it facilitated somehow with this
>>>> testing change?
>>>>
>>>> On Fri, Jun 2, 2017 at 4:33 PM, Dennis Kliban <dkliban at redhat.com>
>>>> wrote:
>>>>
>>>>> In an effort to resolve issue 2751[0], I updated the PR builder job
>>>>> for plugins. Each PR for a plugin will now be tested against the latest
>>>>> stable release of the core found here[1]. This will ensure that the plugin
>>>>> is maintaining compatibility with the latest stable core and that we are
>>>>> only testing one change at a time.
>>>>>
>>>>>
>>>>> [0] https://pulp.plan.io/issues/2751
>>>>> [1] https://repos.fedorapeople.org/pulp/pulp/stable/latest/
>>>>>
>>>>>
>>>>> -Dennis
>>>>>
>>>>> _______________________________________________
>>>>> Pulp-dev mailing list
>>>>> Pulp-dev at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Michael Hrivnak
>>>>
>>>> Principal Software Engineer, RHCE
>>>>
>>>> Red Hat
>>>>
>>>> _______________________________________________
>>>> Pulp-dev mailing list
>>>> Pulp-dev at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20170615/46417230/attachment.htm>