[Pulp-dev] Pulp 3 Release Process Questions

Mon Apr 23 19:22:57 UTC 2018

Travis CI runs a few OSes, but not RHEL and Fedora, or many others. Travis
is good for ensuring that Pulp's main release asset (the pypi packages) are
being tested continuously.

Ensuring that Pulp runs on any given OS is a different kind of testing.
Whoever packages for a given distro (rhel, fedora, debian, etc) will need
to determine how they will test/track the correctness of the packaged asset
itself. To that end the build team may want to run pulp smash against built
RPMs in a different environment other than Travis and that would be fine.

On Mon, Apr 23, 2018 at 2:27 PM, Preethi Thomas <pthomas at redhat.com> wrote:

>
>
> On Mon, Apr 23, 2018 at 2:01 PM, Robin Chan <rchan at redhat.com> wrote:
>
>> I feel like the Travis change is recent enough that I'm not exactly sure
>> how they differ from the Jenkins jobs. Are we all clear on these
>> terminology? Aren't there multiple jobs running at different times? I am
>> not comfortable enough without the assurance that we are all talking about
>> the same things to say we don't need "the Jenkins jobs". I would like for
>> some of these discussions around release process and where (and what and
>> when)  different tests are run be a little bit more collaborative from the
>> beginning rather than reactive post-change happening.
>>
>> I'm all for making quick incremental changes. I'm happy that there are
>> test results that are accessible to everyone. I don't want us to over
>> thinking these, but are we sure we don't have any tests that need to stay
>> private (do we test for any security type things that might need to be
>> embargoed before reported)? I want to make sure we think thing through
>> before we start tearing stuff down.
>>
>> On Mon, Apr 23, 2018 at 1:37 PM, Brian Bouterse <bbouters at redhat.com>
>> wrote:
>>
>>>
>>> On Mon, Apr 23, 2018 at 1:25 PM, David Davis <daviddavis at redhat.com>
>>> wrote:
>>>
>>>> > Are those Travis jobs testing combinations of web servers, AMQP
>>>> brokers, databases, etc? If not, is testing across those combinations a
>>>> goal?
>>>>
>>>> This is definitely the goal. Right now, our build matrix tests against
>>>> different databases, different Django versions, and different python
>>>> versions. We could look at testing against different webservers but right
>>>> now we just use django’s built-in webserver. AMQP is going away though[0].
>>>>
>>>> > Let's say pulp_file is being tested. Is both it and pulpcore
>>>> installed from source? Or is pulpcore installed from egg/wheel?
>>>>
>>>> Yes, we’re testing against source which also allows us to test against
>>>> different PRs. Often, a pulpcore change will require a pulp_file change or
>>>> vice versa and we just added some functionality recently that actually lets
>>>> us test against PRs too.[1] pulp-smash is also running against source or
>>>> PRs. IMO, testing against source lets us move faster than testing against
>>>> releases. Maybe we can look at switching though once things have stabilized
>>>> after the MVP.
>>>>
>>>> > Who is looking at Jenkins jobs? If only QE, why have Jenkins jobs?
>>>>
>>>> I don’t think we need them. We can trigger builds on an as-needed basis
>>>> with Travis without making git commits. We’re also running builds nightly
>>>> right now with Travis’ cron.
>>>>
>>>
>>> I agree we don't need the Jenkins jobs for Pulp3 and they can be
>>> deleted. We still need to keep them for Pulp2 because Pulp2 can't run on
>>> Travis. Moving to Travis is a big step forward for the community since
>>> Travis is readable by the whole Internet while to read Jenkins jobs, you
>>> have to be a Red Hat employee.
>>>
>>
> I admit that I know very little about Travis. So I did some reading up here
>
> https://docs.travis-ci.com/user/reference/overview/#For-
> a-particular-.travis.yml-configuration
>
> Does Travis-ci run only on Ubuntu?
>
> If that is is true, how are we planning to test fedora/centos/rhel ? With
> our history of SeLinux issues in pulp2 I would think we need to be running
> tests in these as well.
>
>
>
>>
>>>
>>>>
>>>> > What does this mean for goals like multi-host testing?
>>>>
>>>> It looks like Travis supports running docker containers[2] so maybe we
>>>> could leverage containers to test this sort of setup. There are definitely
>>>> limitations of Travis that might require us to move to something more
>>>> robust in the future but short-term I think/hope Travis will fit our needs.
>>>>
>>>> [0] https://github.com/pulp/pulp/pull/3454
>>>> [1] https://github.com/pulp/pulp/pull/3435
>>>> [2] https://docs.travis-ci.com/user/docker/
>>>>
>>>>
>>>> David
>>>>
>>>> On Mon, Apr 23, 2018 at 12:10 PM, Jeremy Audet <jaudet at redhat.com>
>>>> wrote:
>>>>
>>>>> Using Travis jobs instead of Jenkins jobs for testing Pulp 3 releases
>>>>> begs several questions:
>>>>>
>>>>>    - Are those Travis jobs testing combinations of web servers, AMQP
>>>>>    brokers, databases, etc? If not, is testing across those combinations a
>>>>>    goal?
>>>>>    - Let's say pulp_file is being tested. Is both it and pulpcore
>>>>>    installed from source? Or is pulpcore installed from egg/wheel?
>>>>>    - Who is looking at Jenkins jobs? If only QE, why have Jenkins
>>>>>    jobs? (Possible answer: So that we can trigger test runs on an as-needed
>>>>>    basis, without making new git commits.)
>>>>>    - What does this mean for goals like multi-host testing? That is,
>>>>>    what does this mean for the case where a single Pulp 3 application is
>>>>>    deployed across multiple hosts? Can that be tested with Travis? Pulp Smash
>>>>>    is architected with this in mind, and it's been a long standing goal. The
>>>>>    biggest impediment has been the use of legacy Jenkins (1.x) and nodepool,
>>>>>    and recent upgrades make this a more realistic testing target.
>>>>>
>>>>> If these questions have already been considered and answered
>>>>> somewhere, please point me to the correct resource.
>>>>>
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com
>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>
>>>
>>
>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20180423/57a32c76/attachment.htm>