[Pulp-dev] How to enable HTTPS for our tests in pulpcore and all plugins?

Fri May 7 15:39:10 UTC 2021

On Fri, May 7, 2021 at 11:27 AM Robin Chan <rchan at redhat.com> wrote:

> Can someone enlighten me on the main motivation for making this change?
> I wasn't at the meeting and just curious what other context I'm missing. I
> definitely understand https > http from a security standpoint but wondering
> if there were other factors or motivations I'm missing.
>
It's a good question. I have two main ones, but none are especially
timeline driven:

* it's problematic for development today. The installer (which installs dev
envs also) default to https, but the tests are incompatible with that and
can only work with http. Even though we work with it everyday we regularly
have test failures and spend hours only to realize our local tests aren't
working because we forgot to "unconfigure https" manually. This happened to
me on Tuesday for example. Non-daily-developers would have no way of
knowing this.

* user security: When demoing pulp-ansible with the CLI and container
installs at fosdem for example, the first thing we have to do is instruct
users to disable security.

Maybe others have other reasons too, but those were my interests.

> -rchan
>
> On Fri, May 7, 2021 at 10:53 AM David Davis <daviddavis at redhat.com> wrote:
>
>> To confirm, the "latest" tag will continue to ship with http? I imagine
>> most users will end up with http then.
>>
>> Also, what (if anything) do we do about y release tags (e.g. the upcoming
>> 3.13 tag)? Do they continue to ship with http?
>>
>> David
>>
>>
>> On Fri, May 7, 2021 at 10:51 AM Brian Bouterse <bmbouter at redhat.com>
>> wrote:
>>
>>> awwww yisssss
>>>
>>> On Fri, May 7, 2021 at 10:46 AM Fabricio Aguiar <faguiard at redhat.com>
>>> wrote:
>>>
>>>> I changed https://github.com/pulp/pulp-oci-images/pull/73 to ship both,
>>>> latest as is, and the new tag: https
>>>>
>>>> Best regards,
>>>> Fabricio Aguiar
>>>> Software Engineer, Pulp Project
>>>> Red Hat Brazil - Latam <https://www.redhat.com/>
>>>> +55 22 999000595
>>>>
>>>>
>>>>
>>>> On Fri, May 7, 2021 at 11:41 AM Brian Bouterse <bmbouter at redhat.com>
>>>> wrote:
>>>>
>>>>> +1 to this observation, we probably need to either ship both or make
>>>>> it configurable somehow. Shipping both is probably easier on users.
>>>>>
>>>>> On Fri, May 7, 2021 at 5:11 AM Matthias Dellweg <mdellweg at redhat.com>
>>>>> wrote:
>>>>>
>>>>>> This is a great piece of work!
>>>>>> The problem I see is that the SSL free container image may be used in
>>>>>> places we do not control. And having this http based container equipped
>>>>>> with an external https reverse proxy is imho a valid use case.
>>>>>> Therefore i would prefer, if we could provide both versions of the
>>>>>> image (with and without SSL) as different tags.
>>>>>> This would also give us the opportunity to switch the plugins one by
>>>>>> one to use the new container.
>>>>>> Ideally, the SSL container would be a thin OCI-layer on top of the
>>>>>> http version.
>>>>>>
>>>>>> On Thu, May 6, 2021 at 10:10 PM Fabricio Aguiar <faguiard at redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I finally made pulp_container CI work with https,
>>>>>>> I also did some changes on pulp_installer, I believe these changes
>>>>>>> will make it possible to run functional tests on dev environment.
>>>>>>>
>>>>>>> I think now it is a matter of deciding when is the best time to
>>>>>>> merge the PR on the single container and if latest tag should be https or
>>>>>>> not
>>>>>>>
>>>>>>> PRs:
>>>>>>> https://github.com/pulp/pulp-oci-images/pull/73
>>>>>>> https://github.com/pulp/pulp_installer/pull/614
>>>>>>> https://github.com/pulp/plugin_template/pull/379
>>>>>>> https://github.com/pulp/pulpcore/pull/1283
>>>>>>> https://github.com/pulp/pulp_container/pull/304
>>>>>>> https://github.com/pulp/pulp_rpm/pull/1977
>>>>>>> https://github.com/pulp/pulp_ansible/pull/572
>>>>>>> https://github.com/pulp/pulp-2to3-migration/pull/362
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Fabricio Aguiar
>>>>>>> Software Engineer, Pulp Project
>>>>>>> Red Hat Brazil - Latam <https://www.redhat.com/>
>>>>>>> +55 22 999000595
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Apr 27, 2021 at 5:35 PM Fabricio Aguiar <faguiard at redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I created https branch:
>>>>>>>> https://github.com/pulp/pulp-oci-images/tree/https
>>>>>>>> and pushed the following images:
>>>>>>>> - pulp/pulp-ci-centos:https
>>>>>>>> - pulp/pulp:https
>>>>>>>>
>>>>>>>> Now we can test on the plugins,
>>>>>>>> I followed your suggestion and did it on pulp_npm:
>>>>>>>> https://github.com/pulp/pulp_npm/pull/89
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>> Fabricio Aguiar
>>>>>>>> Software Engineer, Pulp Project
>>>>>>>> Red Hat Brazil - Latam <https://www.redhat.com/>
>>>>>>>> +55 22 999000595
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Apr 27, 2021 at 9:25 AM David Davis <daviddavis at redhat.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> This is great. Thank you for working on it.
>>>>>>>>>
>>>>>>>>> As a next step, would it make sense to create a branch and then
>>>>>>>>> try to deploy a new temporary tag from that branch? Then maybe we can test
>>>>>>>>> a plugin (eg pulp_npm) against this new image and see what breaks.
>>>>>>>>>
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Apr 26, 2021 at 5:01 PM Fabricio Aguiar <
>>>>>>>>> faguiard at redhat.com> wrote:
>>>>>>>>>
>>>>>>>>>> I started this POC:
>>>>>>>>>> https://github.com/pulp/pulp-oci-images/pull/73
>>>>>>>>>> It enables https on the single container, once merged, the CI for
>>>>>>>>>> every plugin will run the functional tests using https.
>>>>>>>>>> Probably it would break the majority of the CIs, we need to
>>>>>>>>>> discuss when is the best moment to merge this PR or discuss alternatives
>>>>>>>>>>
>>>>>>>>>> Best regards,
>>>>>>>>>> Fabricio Aguiar
>>>>>>>>>> Software Engineer, Pulp Project
>>>>>>>>>> Red Hat Brazil - Latam <https://www.redhat.com/>
>>>>>>>>>> +55 22 999000595
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Feb 9, 2021 at 10:55 AM Fabricio Aguiar <
>>>>>>>>>> faguiard at redhat.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Our nginx conf only supports http now:
>>>>>>>>>>> https://github.com/pulp/pulp-oci-images/blob/latest/assets/nginx.conf#L15
>>>>>>>>>>> For not breaking all plugins, I believe we can build a new CI
>>>>>>>>>>> image that supports https.
>>>>>>>>>>> Maybe a template_config parameter - test_https: true would
>>>>>>>>>>> switch the images
>>>>>>>>>>>
>>>>>>>>>>> Best regards,
>>>>>>>>>>> Fabricio Aguiar
>>>>>>>>>>> Software Engineer, Pulp Project
>>>>>>>>>>> Red Hat Brazil - Latam <https://www.redhat.com/>
>>>>>>>>>>> +55 22 999000595
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Feb 9, 2021 at 5:16 AM Matthias Dellweg <
>>>>>>>>>>> mdellweg at redhat.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I believe this is at least solving the problem partially:
>>>>>>>>>>>>
>>>>>>>>>>>> https://github.com/pulp/pulp-smash/pull/1251
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Feb 8, 2021 at 9:48 PM Brian Bouterse <
>>>>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I believe all of our plugins (and CI) require HTTP and do not
>>>>>>>>>>>>> work with HTTPS. I'm not well versed in what needs to be done to fix this,
>>>>>>>>>>>>> but I think we should fix it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Can the CI group have a 30 min call to talk over what needs to
>>>>>>>>>>>>> be done? Or maybe share some info here?
>>>>>>>>>>>>>
>>>>>>>>>>>>> The main issue I'm aware of is that the tests are not prepared
>>>>>>>>>>>>> to trust an https certificate that is self-signed. I'm not exactly sure
>>>>>>>>>>>>> where we can change that in one place either.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>> Brian
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> Pulp-dev mailing list
>>>>>>>>>>>>> Pulp-dev at redhat.com
>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> Pulp-dev mailing list
>>>>>>>>>>>> Pulp-dev at redhat.com
>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>> Pulp-dev mailing list
>>>>>>>>>> Pulp-dev at redhat.com
>>>>>>>>>> https://listman.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>> Pulp-dev mailing list
>> Pulp-dev at redhat.com
>> https://listman.redhat.com/mailman/listinfo/pulp-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20210507/ffc1cfcf/attachment.htm>