[Pulp-dev] Announcing the plugin-template change to use containers for CI

[Mike DePaulo]

*Why?*
pulp_rpm, and likely other plugins in the future, have C dependencies. They
are often difficult to satisfy on Ubuntu's LTS releases that Travis
provides. Since we are developing containers and a Kubernetes operator
anyway, we decided to migrate plugin-template's Travis CI scripts to using
them. These containers are currently based on Fedora 30, but will likely
later be based on CentOS 7.7 or 8.0, which contain python 3.6.
This is task #5004 <https://pulp.plan.io/issues/5004>.

*Overview:*
1. During install.sh, the container image named like "pulp_file" is built
according to pulpcore/containers/
2. During install.sh, k3s, the lightweight kubernetes implementation, is
installed & configured on the Travis Ubuntu VM, which already has docker.
3. During install.sh, pulp-operator is deployed (up.sh), which in turn
deploys the postgres database, redis, and the pulp_* image you built. It
spins the single image up 4 separate containers (services/processes):
pulp-api, pulp-content, pulp-resource-manager, and pulp-worker .
4. During script.sh, we install some commonly needed testing tools into the
pulp-api container by prefixing the install command with $CMD_PREFIX . So
long as this container never gets restarted, they will remain installed in
it. These tools are not available to the other containers.
5. We use $CMD_PREFIX to run the unit tests
6. From the Travis Ubuntu VM, we run pulp-smash functional tests, build
docs, or use bindings, etc.
7. Note that many other pieces of the puzzle were modified, including
pulp-smash, to make this integration all work.

*Changes you may need to make:*
(in addition to the usual plugin-template commands):
1. Ensure that in template_config.yml, "plugin_name" is set to a value like
"pulp_file" rather than "file"
2. If you write out ~/.netrc , the pulp application password has been
changed from "admin" to "password" (which other pulp test / development
code uses.)
3. git rm .travis/{playbook,postgres,mariadb}.yml

*Planned improvements include:*
1. Speeding up in the image build, largely through 3.
2. Publishing images to quay.io. #5062 <https://pulp.plan.io/issues/5062>
3. Only building your pulp_* image from scratch if you have a required PR
for pulpcore or pulpcore-plugin (or pulp-certguard.) Otherwise, your image
will be layered on top of the "pulpcore" image. #5062
<https://pulp.plan.io/issues/5062>
4. re-introducing python 3.6 testing alongside 3.7 (Fedora 30 is Python 3.7
based)
5. re-introducing codecov
6. (possible / long-term): Improving how testing packages are installed. #5404
<https://pulp.plan.io/issues/5404>

*Feedback wanted on:*
1. Any confusing/unclear output.
2. Any reliability issues.


-- 

Mike DePaulo

He / Him / His

Service Reliability Engineer, Pulp

Red Hat <https://www.redhat.com/>

IM: mikedep333

GPG: 51745404
<https://www.redhat.com/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/pulp-dev/attachments/20190904/cd6bfd5c/attachment.htm>