[Pulp-dev] Pulp 3 release quality

Tue Apr 17 20:22:50 UTC 2018

bmbouter is fixing the docs link in a PR: https://github.com/pulp/pulp/
pull/3446

My understanding is that the smoke tests are for PRs, but actual releases
would still be tested against the full test suite.

On Tue, Apr 17, 2018 at 3:25 PM, Preethi Thomas <pthomas at redhat.com> wrote:

>
>
> On Tue, Apr 17, 2018 at 8:32 AM, David Davis <daviddavis at redhat.com>
> wrote:
>
>> A couple things. First, we’re only running pulpcore and pulp_file tests
>> against pulpcore PRs. Also, we’re not running all pulp-smash tests for each
>> PR—only a certain subset labeled as "smoke tests". That said, we’ll still
>> want to keep an eye on pulp-smash tests over time to make sure it doesn’t
>> become a bottleneck.
>>
>
> This raises another concern.
>
> If the "smoke tests" are the only tests run before releasing to PyPi, what
> would be plan to run the complete integration test suite?
>
> Also, looking at
> https://docs.pulpproject.org/en/3.0/nightly/contributing/con
> tinuous_integration.html
>
> "The unit tests for both pulpcore and pulpcore-plugin live in the tests
> <https://github.com/pulp/pulp/blob/3.0-dev/tests/> folder"
>
> The link seems to rendering a 404
>
>
>>
>> David
>>
>> On Tue, Apr 17, 2018 at 8:24 AM, Preethi Thomas <pthomas at redhat.com>
>> wrote:
>>
>>> "The plan for quality involves the continuous delivery of Pulp 3 where
>>> both unit and integration are run with each PR and prior to each release to
>>> PyPI"
>>>
>>> My concern about the above statement is that as the number of tests
>>> increases, the time it will take to run the pulp-smash integration suite
>>> will also increase. If each PR is required to run the integration suite
>>> then pulp-smash will become the bottleneck.
>>>
>>> Just for comparison purpose, it takes about 2.5 hours to run pulp-smash
>>> for Pulp 2.
>>>
>>>
>>>
>>> On Mon, Apr 16, 2018 at 4:30 PM, Dennis Kliban <dkliban at redhat.com>
>>> wrote:
>>>
>>>> The documentation has been updated[0].
>>>>
>>>> [0] https://docs.pulpproject.org/en/3.0/nightly/contributing/con
>>>> tinuous_integration.html
>>>>
>>>> On Fri, Apr 6, 2018 at 2:52 PM, Dennis Kliban <dkliban at redhat.com>
>>>> wrote:
>>>>
>>>>> I have updated the redmine issue[0] to include a page in the
>>>>> contributors guide on "Continuous Integration" as a deliverable. That page
>>>>> will include that quote.
>>>>>
>>>>>
>>>>> [0] https://pulp.plan.io/issues/3530
>>>>>
>>>>> On Fri, Apr 6, 2018 at 2:06 PM, Robin Chan <rchan at redhat.com> wrote:
>>>>>
>>>>>> Dennis,
>>>>>>
>>>>>> Thanks for putting this together. I don't see any responses on this
>>>>>> thread and take that to mean there were no concerns about this proposal.
>>>>>>
>>>>>> Would this process/responsibility change need to go anywhere? (Side
>>>>>> questions, was this technically a PUP?)
>>>>>> "author of the PR would need to be responsible for making additional
>>>>>> PRs to pulp_file and/or pulp-smash to fix the smash tests"
>>>>>>
>>>>>> I think this is minor enough to skip some process, however I would
>>>>>> like to see the above quote go into the developers guide. A new contributor
>>>>>> shouldn't have to glean through mail list archives to understand this new
>>>>>> responsibility.
>>>>>>
>>>>>> Thanks,
>>>>>> Robin
>>>>>>
>>>>>> On Tue, Mar 27, 2018 at 8:22 PM, Dennis Kliban <dkliban at redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>> One of the requirements for this plan to work is making sure that
>>>>>>> pulp, pulp_file, and pulp-smash always continue working together. This
>>>>>>> means that any time we have a PR that breaks pulp-smash tests, the author
>>>>>>> of the PR would need to be responsible for making additional PRs to
>>>>>>> pulp_file and/or pulp-smash to fix the smash tests. To enforce this
>>>>>>> requirement using Travis, I have filed 2 tasks[0,1] in redmine that I would
>>>>>>> like to get groomed and added to the sprint. I plan on working on these
>>>>>>> tasks as I introduce a change that will result from resolving issue 3488[2].
>>>>>>>
>>>>>>> [0] https://pulp.plan.io/issues/3530
>>>>>>> [1] https://pulp.plan.io/issues/3531
>>>>>>> [2] https://pulp.plan.io/issues/3488
>>>>>>>
>>>>>>> On Fri, Mar 23, 2018 at 2:37 PM, Dennis Kliban <dkliban at redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I've started putting together a Continuous Delivery of Pulp 3
>>>>>>>> page[0] on our wiki.
>>>>>>>>
>>>>>>>> This page outlines a plan for how we can ensure and prove the
>>>>>>>> quality of Pulp 3 releases by relying on pulp-smash tests and unit tests.
>>>>>>>>
>>>>>>>> This plan enables anyone to improve the quality of Pulp 3 releases
>>>>>>>> through contributions to pulp-smash and unit tests.
>>>>>>>>
>>>>>>>> Please take a look at the plan and provide feedback on this thread
>>>>>>>> or feel free to make edits directly on the page.
>>>>>>>>
>>>>>>>> [0] https://pulp.plan.io/projects/pulp/wiki/Continuous_Delivery_
>>>>>>>> of_Pulp_3
>>>>>>>>
>>>>>>>> -Dennis
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Pulp-dev mailing list
>>>>>>> Pulp-dev at redhat.com
>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Pulp-dev mailing list
>>>> Pulp-dev at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>>
>>
>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180417/43a95cdc/attachment.htm>