[Pulp-dev] How to enable HTTPS for our tests in pulpcore and all plugins?

Fri Jul 30 13:54:45 UTC 2021

Currently, pulp-oci-images is building:
*Tag* *Scheme*
latest http
https https

CI is running tests on https images (https image is python38),
if you experience failures when your run tests on your dev environment,
please make sure pulp_webserver_disable_https is false or commented on your
local.dev-config.yml
https://github.com/pulp/pulp_installer/blob/master/example.dev-config.yml#L48-L49

Best regards,
Fabricio Aguiar
Software Engineer, Pulp Project
Red Hat Brazil - Latam <https://www.redhat.com/>
+55 22 999000595

On Fri, May 14, 2021 at 11:14 AM Matthias Dellweg <mdellweg at redhat.com>
wrote:

> Tags in the container world are cheap. Let's add a "http" tag that points
> to the same image as latest.
> I think, we should additionally provide the released images as an https
> version maybe tagged "x.y-https", but this can/should be postponed. Let's
> first get comfy with ssl in the latest build.
>
> On Fri, May 14, 2021 at 4:06 PM Fabricio Aguiar <faguiard at redhat.com>
> wrote:
>
>> Bump!
>>
>> Single container PR [1] needs some adjustments, I plan to address them
>> once we decide about the tags.
>> Current PR makes:
>> *Tag* *Scheme*
>> latest http
>> https https
>> x.y http
>>
>> Please share your feedback about the tag/scheme until May 19
>>
>> [1] https://github.com/pulp/pulp-oci-images/pull/73
>>
>> Best regards,
>> Fabricio Aguiar
>> Software Engineer, Pulp Project
>> Red Hat Brazil - Latam <https://www.redhat.com/>
>> +55 22 999000595
>>
>>
>>
>> On Mon, May 10, 2021 at 9:07 AM Ina Panova <ipanova at redhat.com> wrote:
>>
>>> I would get rid of the latest tag because it is non-deterministic and
>>> would keep http/https tags only.
>>>
>>> --------
>>> Regards,
>>>
>>> Ina Panova
>>> Senior Software Engineer| Pulp| Red Hat Inc.
>>>
>>> "Do not go where the path may lead,
>>>  go instead where there is no path and leave a trail."
>>>
>>>
>>> On Fri, May 7, 2021 at 6:08 PM Matthias Dellweg <mdellweg at redhat.com>
>>> wrote:
>>>
>>>> I would tag http and https and then latest as the same as http. Then we
>>>> can write an announcement that we will switch latest from http to https or
>>>> drop latest altogether.
>>>> The question about release tags is a good one. I think, we need both
>>>> there too.
>>>>
>>>> On Fri, May 7, 2021 at 6:05 PM David Davis <daviddavis at redhat.com>
>>>> wrote:
>>>>
>>>>> I feel like ideally, https would be the default (ie latest). However,
>>>>> then we are going to break all the release branches for pulpcore and
>>>>> plugins that are pointing to latest but not expecting https.
>>>>>
>>>>> Hopefully people will weigh in here.
>>>>>
>>>>> David
>>>>>
>>>>>
>>>>> On Fri, May 7, 2021 at 11:55 AM Fabricio Aguiar <faguiard at redhat.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, May 7, 2021 at 11:52 AM David Davis <daviddavis at redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>> To confirm, the "latest" tag will continue to ship with http? I
>>>>>>> imagine most users will end up with http then.
>>>>>>>
>>>>>> I can modify the PR and make https the default
>>>>>>
>>>>>>>
>>>>>>> Also, what (if anything) do we do about y release tags (e.g. the
>>>>>>> upcoming 3.13 tag)? Do they continue to ship with http?
>>>>>>>
>>>>>> I think release tags can be https
>>>>>>
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>>
>>>>>>> On Fri, May 7, 2021 at 10:51 AM Brian Bouterse <bmbouter at redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> awwww yisssss
>>>>>>>>
>>>>>>>> On Fri, May 7, 2021 at 10:46 AM Fabricio Aguiar <
>>>>>>>> faguiard at redhat.com> wrote:
>>>>>>>>
>>>>>>>>> I changed https://github.com/pulp/pulp-oci-images/pull/73 to ship
>>>>>>>>> both,
>>>>>>>>> latest as is, and the new tag: https
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>> Fabricio Aguiar
>>>>>>>>> Software Engineer, Pulp Project
>>>>>>>>> Red Hat Brazil - Latam <https://www.redhat.com/>
>>>>>>>>> +55 22 999000595
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, May 7, 2021 at 11:41 AM Brian Bouterse <
>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>
>>>>>>>>>> +1 to this observation, we probably need to either ship both or
>>>>>>>>>> make it configurable somehow. Shipping both is probably easier on users.
>>>>>>>>>>
>>>>>>>>>> On Fri, May 7, 2021 at 5:11 AM Matthias Dellweg <
>>>>>>>>>> mdellweg at redhat.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> This is a great piece of work!
>>>>>>>>>>> The problem I see is that the SSL free container image may be
>>>>>>>>>>> used in places we do not control. And having this http based container
>>>>>>>>>>> equipped with an external https reverse proxy is imho a valid use case.
>>>>>>>>>>> Therefore i would prefer, if we could provide both versions of
>>>>>>>>>>> the image (with and without SSL) as different tags.
>>>>>>>>>>> This would also give us the opportunity to switch the plugins
>>>>>>>>>>> one by one to use the new container.
>>>>>>>>>>> Ideally, the SSL container would be a thin OCI-layer on top of
>>>>>>>>>>> the http version.
>>>>>>>>>>>
>>>>>>>>>>> On Thu, May 6, 2021 at 10:10 PM Fabricio Aguiar <
>>>>>>>>>>> faguiard at redhat.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I finally made pulp_container CI work with https,
>>>>>>>>>>>> I also did some changes on pulp_installer, I believe these
>>>>>>>>>>>> changes will make it possible to run functional tests on dev environment.
>>>>>>>>>>>>
>>>>>>>>>>>> I think now it is a matter of deciding when is the best time to
>>>>>>>>>>>> merge the PR on the single container and if latest tag should be https or
>>>>>>>>>>>> not
>>>>>>>>>>>>
>>>>>>>>>>>> PRs:
>>>>>>>>>>>> https://github.com/pulp/pulp-oci-images/pull/73
>>>>>>>>>>>> https://github.com/pulp/pulp_installer/pull/614
>>>>>>>>>>>> https://github.com/pulp/plugin_template/pull/379
>>>>>>>>>>>> https://github.com/pulp/pulpcore/pull/1283
>>>>>>>>>>>> https://github.com/pulp/pulp_container/pull/304
>>>>>>>>>>>> https://github.com/pulp/pulp_rpm/pull/1977
>>>>>>>>>>>> https://github.com/pulp/pulp_ansible/pull/572
>>>>>>>>>>>> https://github.com/pulp/pulp-2to3-migration/pull/362
>>>>>>>>>>>>
>>>>>>>>>>>> Best regards,
>>>>>>>>>>>> Fabricio Aguiar
>>>>>>>>>>>> Software Engineer, Pulp Project
>>>>>>>>>>>> Red Hat Brazil - Latam <https://www.redhat.com/>
>>>>>>>>>>>> +55 22 999000595
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Apr 27, 2021 at 5:35 PM Fabricio Aguiar <
>>>>>>>>>>>> faguiard at redhat.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I created https branch:
>>>>>>>>>>>>> https://github.com/pulp/pulp-oci-images/tree/https
>>>>>>>>>>>>> and pushed the following images:
>>>>>>>>>>>>> - pulp/pulp-ci-centos:https
>>>>>>>>>>>>> - pulp/pulp:https
>>>>>>>>>>>>>
>>>>>>>>>>>>> Now we can test on the plugins,
>>>>>>>>>>>>> I followed your suggestion and did it on pulp_npm:
>>>>>>>>>>>>> https://github.com/pulp/pulp_npm/pull/89
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>> Fabricio Aguiar
>>>>>>>>>>>>> Software Engineer, Pulp Project
>>>>>>>>>>>>> Red Hat Brazil - Latam <https://www.redhat.com/>
>>>>>>>>>>>>> +55 22 999000595
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Apr 27, 2021 at 9:25 AM David Davis <
>>>>>>>>>>>>> daviddavis at redhat.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> This is great. Thank you for working on it.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As a next step, would it make sense to create a branch and
>>>>>>>>>>>>>> then try to deploy a new temporary tag from that branch? Then maybe we can
>>>>>>>>>>>>>> test a plugin (eg pulp_npm) against this new image and see what breaks.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> David
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Apr 26, 2021 at 5:01 PM Fabricio Aguiar <
>>>>>>>>>>>>>> faguiard at redhat.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I started this POC:
>>>>>>>>>>>>>>> https://github.com/pulp/pulp-oci-images/pull/73
>>>>>>>>>>>>>>> It enables https on the single container, once merged, the
>>>>>>>>>>>>>>> CI for every plugin will run the functional tests using https.
>>>>>>>>>>>>>>> Probably it would break the majority of the CIs, we need to
>>>>>>>>>>>>>>> discuss when is the best moment to merge this PR or discuss alternatives
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>> Fabricio Aguiar
>>>>>>>>>>>>>>> Software Engineer, Pulp Project
>>>>>>>>>>>>>>> Red Hat Brazil - Latam <https://www.redhat.com/>
>>>>>>>>>>>>>>> +55 22 999000595
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Feb 9, 2021 at 10:55 AM Fabricio Aguiar <
>>>>>>>>>>>>>>> faguiard at redhat.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Our nginx conf only supports http now:
>>>>>>>>>>>>>>>> https://github.com/pulp/pulp-oci-images/blob/latest/assets/nginx.conf#L15
>>>>>>>>>>>>>>>> For not breaking all plugins, I believe we can build a new
>>>>>>>>>>>>>>>> CI image that supports https.
>>>>>>>>>>>>>>>> Maybe a template_config parameter - test_https: true would
>>>>>>>>>>>>>>>> switch the images
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Best regards,
>>>>>>>>>>>>>>>> Fabricio Aguiar
>>>>>>>>>>>>>>>> Software Engineer, Pulp Project
>>>>>>>>>>>>>>>> Red Hat Brazil - Latam <https://www.redhat.com/>
>>>>>>>>>>>>>>>> +55 22 999000595
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Feb 9, 2021 at 5:16 AM Matthias Dellweg <
>>>>>>>>>>>>>>>> mdellweg at redhat.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I believe this is at least solving the problem partially:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> https://github.com/pulp/pulp-smash/pull/1251
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Feb 8, 2021 at 9:48 PM Brian Bouterse <
>>>>>>>>>>>>>>>>> bmbouter at redhat.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I believe all of our plugins (and CI) require HTTP and do
>>>>>>>>>>>>>>>>>> not work with HTTPS. I'm not well versed in what needs to be done to fix
>>>>>>>>>>>>>>>>>> this, but I think we should fix it.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Can the CI group have a 30 min call to talk over what
>>>>>>>>>>>>>>>>>> needs to be done? Or maybe share some info here?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The main issue I'm aware of is that the tests are not
>>>>>>>>>>>>>>>>>> prepared to trust an https certificate that is self-signed. I'm not exactly
>>>>>>>>>>>>>>>>>> sure where we can change that in one place either.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>>>>>> Brian
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>> Pulp-dev mailing list
>>>>>>>>>>>>>>>>>> Pulp-dev at redhat.com
>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>> Pulp-dev mailing list
>>>>>>>>>>>>>>>>> Pulp-dev at redhat.com
>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>> Pulp-dev mailing list
>>>>>>>>>>>>>>> Pulp-dev at redhat.com
>>>>>>>>>>>>>>> https://listman.redhat.com/mailman/listinfo/pulp-dev
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>> Pulp-dev mailing list
>>>> Pulp-dev at redhat.com
>>>> https://listman.redhat.com/mailman/listinfo/pulp-dev
>>>>
>>> _______________________________________________
>>> Pulp-dev mailing list
>>> Pulp-dev at redhat.com
>>> https://listman.redhat.com/mailman/listinfo/pulp-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20210730/931e27ff/attachment.htm>