[almighty] CI infrastructure update

[Pavol Pitonak]

Hi,

I'd like to inform team about status quo and future plans regarding our
continuous integration infrastructure.

Almighty team uses internal instance of Jenkins hosted on Red Hat
infrastructure internally referred to as Central CI, CCI or Central CI
Jenkins. Contact person is Pavol Srna, Martin Malina or me.

Jenkins master is managed by ops team and runs in a VM, slaves are managed
by QE team and are provisioned in OpenStack 7 instance. At the moment we
have resource quota for provisioning 10 relatively powerful (4 vcpus, 8gb
ram) slaves that are shared by Almighty and JBoss Tools teams.

All Almightly Jenkins jobs use Docker but unfortunately there is only 1
Jenkins slave with Docker at the moment.

Short term plan (possible to implement immediately):
* Unify all RHEL7 slaves to just one type that would have Docker installed.
There is one issue, though. Jenkins job for almighty-core builds a custom
Docker image which takes quite a lot of time therefore we decided to
provision a dedicated Jenkins slave so that Docker cache wouldn't be pruned
too often.

If we want to use our resources in the most effective way possible, we need
to provision slaves dynamically (up to some limit) and then destroy them
when not used. I would need you to split building this interim Docker image
and building Almighty-core into two separate jobs. Then almighty-core job
could run on whatever Jenkins slave relatively fast.

After you do it, we would just update Jenkins configuration for slaves
provisioning (5 minutes of work).

Long term plan (no ETA right now):
* We asked for our quota to be increased significantly, should at least
double at the beginning.
* Jenkins master will be upgraded from RHEL6 to RHEL7, no direct impact on
Almighty team.
* Jenkins will be upgraded to newer LTS version (1.651.3). There was
interest in new Jenkins Pipeline plugins - this plugin will be supported in
this new version of Jenkins.
* Define requirements for Jenkins slaves - OS, minimum CPUs, minimum RAM,
installed software - both for what is required now and what will be
necessary in the future (e.g. testing on multiple operating systems,
multiple versions of ???)
* Right now community cannot see Jenkins builds because the Jenkins runs on
internal infrastructure. It's technically possible to publish results to a
public Jenkins instance. ATM it's not clear if this is desired and how to
implement.
* Decide if a dedicated Jenkins master (not shared with other projects)
would be more suitable in the future (IMHO would be too much overhead for
team at the moment)

P.S. We installed Warnings Jenkins plugin

Regards,
Pavol
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/almighty-public/attachments/20160808/4bea00d4/attachment.htm>