[Pulp-dev] How to enable HTTPS for our tests in pulpcore and all plugins?

Fri May 7 15:50:04 UTC 2021

On Fri, May 7, 2021 at 12:40 PM Brian Bouterse <bmbouter at redhat.com> wrote:

>
>
> On Fri, May 7, 2021 at 11:27 AM Robin Chan <rchan at redhat.com> wrote:
>
>> Can someone enlighten me on the main motivation for making this change?
>> I wasn't at the meeting and just curious what other context I'm missing.
>> I definitely understand https > http from a security standpoint but
>> wondering if there were other factors or motivations I'm missing.
>>
> It's a good question. I have two main ones, but none are especially
> timeline driven:
>
> * it's problematic for development today. The installer (which installs
> dev envs also) default to https, but the tests are incompatible with that
> and can only work with http. Even though we work with it everyday we
> regularly have test failures and spend hours only to realize our local
> tests aren't working because we forgot to "unconfigure https" manually.
> This happened to me on Tuesday for example. Non-daily-developers would have
> no way of knowing this.
>
+1 you were faster and explained better than me,
emphasis on non-daily developers, a couple of times people reach to me to
understand why tests were breaking and this was the reason

>
> * user security: When demoing pulp-ansible with the CLI and container
> installs at fosdem for example, the first thing we have to do is instruct
> users to disable security.
>
> Maybe others have other reasons too, but those were my interests.
>
>
>> -rchan
>>
>> On Fri, May 7, 2021 at 10:53 AM David Davis <daviddavis at redhat.com>
>> wrote:
>>
>>> To confirm, the "latest" tag will continue to ship with http? I imagine
>>> most users will end up with http then.
>>>
>>> Also, what (if anything) do we do about y release tags (e.g. the
>>> upcoming 3.13 tag)? Do they continue to ship with http?
>>>
>>> David
>>>
>>>
>>> On Fri, May 7, 2021 at 10:51 AM Brian Bouterse <bmbouter at redhat.com>
>>> wrote:
>>>
>>>> awwww yisssss
>>>>
>>>> On Fri, May 7, 2021 at 10:46 AM Fabricio Aguiar <faguiard at redhat.com>
>>>> wrote:
>>>>
>>>>> I changed https://github.com/pulp/pulp-oci-images/pull/73 to ship
>>>>> both,
>>>>> latest as is, and the new tag: https
>>>>>
>>>>> Best regards,
>>>>> Fabricio Aguiar
>>>>> Software Engineer, Pulp Project
>>>>> Red Hat Brazil - Latam <https://www.redhat.com/>
>>>>> +55 22 999000595
>>>>>
>>>>>
>>>>>
>>>>> On Fri, May 7, 2021 at 11:41 AM Brian Bouterse <bmbouter at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> +1 to this observation, we probably need to either ship both or make
>>>>>> it configurable somehow. Shipping both is probably easier on users.
>>>>>>
>>>>>> On Fri, May 7, 2021 at 5:11 AM Matthias Dellweg <mdellweg at redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>> This is a great piece of work!
>>>>>>> The problem I see is that the SSL free container image may be used
>>>>>>> in places we do not control. And having this http based container equipped
>>>>>>> with an external https reverse proxy is imho a valid use case.
>>>>>>> Therefore i would prefer, if we could provide both versions of the
>>>>>>> image (with and without SSL) as different tags.
>>>>>>> This would also give us the opportunity to switch the plugins one by
>>>>>>> one to use the new container.
>>>>>>> Ideally, the SSL container would be a thin OCI-layer on top of the
>>>>>>> http version.
>>>>>>>
>>>>>>> On Thu, May 6, 2021 at 10:10 PM Fabricio Aguiar <faguiard at redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I finally made pulp_container CI work with https,
>>>>>>>> I also did some changes on pulp_installer, I believe these changes
>>>>>>>> will make it possible to run functional tests on dev environment.
>>>>>>>>
>>>>>>>> I think now it is a matter of deciding when is the best time to
>>>>>>>> merge the PR on the single container and if latest tag should be https or
>>>>>>>> not
>>>>>>>>
>>>>>>>> PRs:
>>>>>>>> https://github.com/pulp/pulp-oci-images/pull/73
>>>>>>>> https://github.com/pulp/pulp_installer/pull/614
>>>>>>>> https://github.com/pulp/plugin_template/pull/379
>>>>>>>> https://github.com/pulp/pulpcore/pull/1283
>>>>>>>> https://github.com/pulp/pulp_container/pull/304
>>>>>>>> https://github.com/pulp/pulp_rpm/pull/1977
>>>>>>>> https://github.com/pulp/pulp_ansible/pull/572
>>>>>>>> https://github.com/pulp/pulp-2to3-migration/pull/362
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Fabricio Aguiar
>>>>>>>> Software Engineer, Pulp Project
>>>>>>>> Red Hat Brazil - Latam <https://www.redhat.com/>
>>>>>>>> +55 22 999000595
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Apr 27, 2021 at 5:35 PM Fabricio Aguiar <
>>>>>>>> faguiard at redhat.com> wrote:
>>>>>>>>
>>>>>>>>> I created https branch:
>>>>>>>>> https://github.com/pulp/pulp-oci-images/tree/https
>>>>>>>>> and pushed the following images:
>>>>>>>>> - pulp/pulp-ci-centos:https
>>>>>>>>> - pulp/pulp:https
>>>>>>>>>
>>>>>>>>> Now we can test on the plugins,
>>>>>>>>> I followed your suggestion and did it on pulp_npm:
>>>>>>>>> https://github.com/pulp/pulp_npm/pull/89
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>> Fabricio Aguiar
>>>>>>>>> Software Engineer, Pulp Project
>>>>>>>>> Red Hat Brazil - Latam <https://www.redhat.com/>
>>>>>>>>> +55 22 999000595
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Apr 27, 2021 at 9:25 AM David Davis <daviddavis at redhat.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> This is great. Thank you for working on it.
>>>>>>>>>>
>>>>>>>>>> As a next step, would it make sense to create a branch and then
>>>>>>>>>> try to deploy a new temporary tag from that branch? Then maybe we can test
>>>>>>>>>> a plugin (eg pulp_npm) against this new image and see what breaks.
>>>>>>>>>>
>>>>>>>>>> David
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Apr 26, 2021 at 5:01 PM Fabricio Aguiar <
>>>>>>>>>> faguiard at redhat.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I started this POC:
>>>>>>>>>>> https://github.com/pulp/pulp-oci-images/pull/73
>>>>>>>>>>> It enables https on the single container, once merged, the CI
>>>>>>>>>>> for every plugin will run the functional tests using https.
>>>>>>>>>>> Probably it would break the majority of the CIs, we need to
>>>>>>>>>>> discuss when is the best moment to merge this PR or discuss alternatives
>>>>>>>>>>>
>>>>>>>>>>> Best regards,
>>>>>>>>>>> Fabricio Aguiar
>>>>>>>>>>> Software Engineer, Pulp Project
>>>>>>>>>>> Red Hat Brazil - Latam <https://www.redhat.com/>
>>>>>>>>>>> +55 22 999000595
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Feb 9, 2021 at 10:55 AM Fabricio Aguiar <
>>>>>>>>>>> faguiard at redhat.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Our nginx conf only supports http now:
>>>>>>>>>>>> https://github.com/pulp/pulp-oci-images/blob/latest/assets/nginx.conf#L15
>>>>>>>>>>>> For not breaking all plugins, I believe we can build a new CI
>>>>>>>>>>>> image that supports https.
>>>>>>>>>>>> Maybe a template_config parameter - test_https: true would
>>>>>>>>>>>> switch the images
>>>>>>>>>>>>
>>>>>>>>>>>> Best regards,
>>>>>>>>>>>> Fabricio Aguiar
>>>>>>>>>>>> Software Engineer, Pulp Project
>>>>>>>>>>>> Red Hat Brazil - Latam <https://www.redhat.com/>
>>>>>>>>>>>> +55 22 999000595
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Feb 9, 2021 at 5:16 AM Matthias Dellweg <
>>>>>>>>>>>> mdellweg at redhat.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I believe this is at least solving the problem partially:
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://github.com/pulp/pulp-smash/pull/1251
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Feb 8, 2021 at 9:48 PM Brian Bouterse <
>>>>>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I believe all of our plugins (and CI) require HTTP and do not
>>>>>>>>>>>>>> work with HTTPS. I'm not well versed in what needs to be done to fix this,
>>>>>>>>>>>>>> but I think we should fix it.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Can the CI group have a 30 min call to talk over what needs
>>>>>>>>>>>>>> to be done? Or maybe share some info here?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The main issue I'm aware of is that the tests are not
>>>>>>>>>>>>>> prepared to trust an https certificate that is self-signed. I'm not exactly
>>>>>>>>>>>>>> sure where we can change that in one place either.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>> Brian
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> Pulp-dev mailing list
>>>>>>>>>>>>>> Pulp-dev at redhat.com
>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Pulp-dev mailing list
>>>>>>>>>>>>> Pulp-dev at redhat.com
>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Pulp-dev mailing list
>>>>>>>>>>> Pulp-dev at redhat.com
>>>>>>>>>>> https://listman.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com
>>> https://listman.redhat.com/mailman/listinfo/pulp-dev
>>>
>> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://listman.redhat.com/mailman/listinfo/pulp-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20210507/d06b9ad3/attachment.htm>