[Pulp-dev] Moving to Github Actions

Tue Feb 11 18:34:46 UTC 2020

Here are my set of thoughts on many things mentioned.

TL;DR: We still need to run CI on CentOS/Fedora, but using cloud instances
of CentOS/Fedora (interacted with via SSH/Ansible from the GHA Ubuntu
client VM) might be preferable to using Fedora CI for certain tests.

1. "We should test GHA via the ansible-pulp related repos now, and then
come up with a thorough & quick schedule to migrate from Travis to GHA
entirely, resources permitting."
I totally agree with this.

2. "We must use Fedora/Centos CI for SELinux policy testing at all, because
Travis & GHA use Ubuntu, whose kernel doesn't support SELinux."
I do not think this is correct. I've researched this, but haven't test it.
SELinux upstream seems to run their CI on Ubuntu:
https://github.com/SELinuxProject/selinux/blob/master/.travis.yml
https://github.com/SELinuxProject/selinux-testsuite/blob/master/.travis.yml#L53
How did they make this work?
My 1 theory is that Ubuntu's kernel has support for both SELinux and
AppArmor, but Travis slims down the image so that AppArmor does not get
enabled like on a default Ubuntu install. So SELinux can be enabled at
runtime.
My 2nd theory is that enough of a shim (the 2nd link on fake-selinuxfs in
particular) is sufficient to avoid reboot.
However, #4 still negates this option.

3. "We must use Fedora/CentOS CI because pulp-certguard's dependencies are
an issue on Ubuntu"
The plugin-template uses our Fedora containers w/ pulp-operator & k3s.
Isn't this sufficient?

4.  "We must use Fedora/Centos CI for SELinux policy testing with pulp-rpm
/ pulp-certguard. Containers will provide the pulp-rpm / pulp-container
deps on ubuntu, but break SELinux testing because SELinux wraps around the
entire container."
This is a bigger concern. My research confirms this, even lxc does not
support SELinux policies *within* the container. Our SELinux policies
currently support the plugins Katello is integrating: pulp-container,
pulp-file, pulp-rpm, and pulp-certguard. We could still do CI testing of
the 1st 2 on Ubuntu though.

5. "Our CI must run be either capable of running entirely on CentOS/Fedora
CI, or have only certain tests run on them."
I am more in favor of the latter, but there is another possible solution
for SELinux testing. We could have an Amazon EC2 account, openstack
account, etc that GHA or Travis calls out to. Ansilbe molecule has drivers
for many cloud compute types:
https://molecule.readthedocs.io/en/2.22/configuration.html#driver
It would create a Fedora / CentOS instance specifically for testing via
cloud APIs, and then run our ansible installer (SSH) against it from the
GHA / Travis instance, then delete it at the end. It would mean that the
Pulp Project would still have no persistent infrastructure.

6. On using Fedora with their Zuul CI instance:
This looks promising, but having read their PDF, I am concerned that
Fedora's instance would be specifically configured for their use case in
ways that we can reconfigure. Hopefully we can. Some of their use case is
integration with pagure rather than GitHub, Koji artifact storage
integration, etc:
https://fedoraproject.org/w/uploads/1/1e/CI_CD_for_Fedora_packaging_with_Zuul_-_final_-_with_notes.pdf

-Mike

On Mon, Feb 10, 2020 at 8:29 AM David Davis <daviddavis at redhat.com> wrote:

> Is this the Software Factory instance of Zuul[0]? I can reach out to them
> and see if it would make sense as an option for Pulp.
>
> [0] https://softwarefactory-project.io/zuul/t/local/status
>
> David
>
>
> On Sun, Feb 9, 2020 at 11:51 AM Neal Gompa <ngompa13 at gmail.com> wrote:
>
>> On Sun, Feb 9, 2020 at 9:46 AM David Davis <daviddavis at redhat.com> wrote:
>>
>>> Thanks Brian and Daniel. I agree on the points you both raised.
>>>
>>> Brian, to you specific questions/points:
>>>
>>> ## We need details on each piece of the Travis workflow, where it will
>>> be ported to, and a rough estimate of how long each piece would take. I
>>> think these things would make a great EPIC.
>>>
>>> I have a Github Actions epic. I plan to update it this week based on our
>>> conversation and will add more specific details, estimates, etc. I'll
>>> respond when it's ready for review.
>>>
>>> https://pulp.plan.io/issues/6065
>>>
>>>
>>> ## Who will work on it? It needs I think 2 fully dedicated people who
>>> already completely understand the Travis stuff in detail. It's too hard for
>>> one person and would take too long...
>>>
>>> I definitely agree we need at least 2 people to work on this. We need as
>>> many people as possible to understand Github Actions.
>>>
>>> I don't know who has time for this right now. I imagine it'll probably
>>> have to wait until next sprint (Sprint 67). Or at least I personally won't
>>> have time until next week at the earliest. That'll give us time to plan
>>> though.
>>>
>>> In the meantime, I'd consider letting the installer team merge
>>> Fabricio's ansible-pulp PR[0]. This will also alleviate much of the
>>> immediate need and let us begin collecting real world data/experience as
>>> well.
>>>
>>>
>> Has anyone reached out to the Fedora CI team about using their Zuul
>> instance? Perhaps they've got an easy automated process for using that on
>> projects. Zuul can spin up either Fedora or CentOS environments, which
>> should satisfy the need for being able to test esoteric things like FIPS
>> mode while also being able to get fresh environments and dependencies
>> through Fedora environments.
>>
>> It may be better to consider using Fedora CI over CentOS CI due to the
>> better system overall, too...
>>
>>
>>
>> --
>> 真実はいつも一つ！/ Always, there's only one truth!
>>
> _______________________________________________
> Pulp-dev mailing list
> Pulp-dev at redhat.com
> https://www.redhat.com/mailman/listinfo/pulp-dev
>

-- 

Mike DePaulo

He / Him / His

Service Reliability Engineer, Pulp

Red Hat <https://www.redhat.com/>

IM: mikedep333

GPG: 51745404
<https://www.redhat.com/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20200211/6b248c83/attachment.htm>