[kontinuity-dev-public] "Build slaves" in Jenkins pipelines

[Tomas Nozicka]

Inline.

On Út, 2016-05-31 at 22:58 -0400, Ben Parees wrote:
> 
> 
> On Tue, May 31, 2016 at 10:27 AM, James Strachan <jstracha at redhat.com
> > wrote:
> > I’m a huge fan of 2); but currently that requires a pod for the
> > JNLP slave and another pod for the docker image; which on premise
> > or on Dedicated doesn’t really matter. But on OpenShift Online that
> > is gonna be painful (3 pods just for builds!); so option 1) using
> > one or two curated pools of slaves of custom pod/mages might be
> > simpler for Online. e.g. one slave pool for maven/java and one for
> > nodejs?
> > 
> yeah so i've been leaning towards (1) due to it needing fewer pods, 
Ok, I agree. Let's go with option 1 for now (and summit) and switch to
option 2 in future when some of the problems are resolved.

> but it does mean we have to provide some slave images, and/or an easy
> way to create new slave images (which we actually already have here:
> https://github.com/openshift/origin/blob/master/examples/jenkins/mast
> er-slave/jenkins-slave-template.json), so certainly (2) feels like a
> better long term solution.
I have tried that template but if I understand it correctly:
 - it uses strategy "Docker" => privileged => won't work on Online and
other secured environments?
 - requires your builder image to be CentOS based (which I am ok with -
people could want something like fedora for newer packages, but they
can always make the full image by themselves)
Otherwise it can be useful.

(Btw. if anyone is trying that template, you need to modify
SLAVE_REPO_CONTEXTDIR 's~examples/jenkins-
master/slave~examples/jenkins/master-slave/slave~' - it used to live in
another directory.)

I guess, we should add this template to ADB, too.

> 
> Providing maven/java and nodejs slave images out of the box seems
> likely.
Yes, when going with option 1, this is almost necessity. We will need
at least a Java/Maven and NodeJS builders. And I was thinking they
should be provided by default when you instantiate the new Jenkins
template. What do you think, Ben?

Or should Catapult be setting them up? The pipelines will be fully
dependent on them.

We should also coordinate on creating those basic images and publish
them to Docker Hub, RH registry. 

Maybe we could instantiate the template in OSE and set it up to push
the images to Docker Hub?

Thanks,
Tomas

> 
>  
> > 
> > 
> > > On 31 May 2016, at 15:19, Tomas Nozicka <tnozicka at redhat.com>
> > > wrote:
> > > 
> > > Hi,
> > > 
> > > how do we want to run our Jenkins "build slaves" in our
> > > pipelines?
> > > 
> > > Options:
> > > 1) kubernetes plugin [1]
> > > = Example =
> > > node ('java8-maven') {
> > >   sh 'mvn package'
> > > }
> > > = EOF =
> > >  - tightly coupled to Jenkins instance (requires predefined pools
> > > of
> > > slaves which you can choose from)
> > >  - build environment is defined by admin not pipeline
> > > (Jenkinsfile)
> > >  - not generic
> > > 
> > > 2) kubernetes-workflow plugin [2]
> > > = Example =
> > > kubernetes.pod('buildpod').withImage('java8-maven').inside{
> > >   sh 'mvn package'
> > > }
> > > = EOF =
> > >  + loosely coupled to Jenkins instance
> > >  + pipeline (Jenkinsfile) specifies build environment
> > >  + generic (pipelines can specify custom images)
> > > 
> > > I am for option 2 because of the loose coupling to Jenkins and
> > > ability
> > > to specify (custom) build images as part of Jenkinsfile.
> > > 
> > > And AFAIK catapult project is partially about taking user
> > > pipelines and
> > > setting those up with OpenShift(+Jenkins), I just don't see it
> > > working
> > > with the tight coupling and option 1.
> > > 
> > > If we decide to go with option 2 we should probably integrate
> > > kubernetes-workflow plugin into jenkins-1-centos7:dev image.
> > > 
> > > What do you think?
> > > 
> > > Thanks,
> > > Tomas
> > > 
> > > [1] - https://wiki.jenkins-ci.org/display/JENKINS/Kubernetes+Plug
> > > in
> > > [2] - https://github.com/fabric8io/kubernetes-workflow
> > > 
> > > _______________________________________________
> > > kontinuity-dev-public mailing list
> > > kontinuity-dev-public at redhat.com
> > > https://www.redhat.com/mailman/listinfo/kontinuity-dev-public
> > 
> > James
> > -------
> > Red Hat
> > 
> > Twitter: @jstrachan
> > Email: jstracha at redhat.com
> > Blog: https://medium.com/@jstrachan/
> > 
> > fabric8: http://fabric8.io/
> > open source microservices platform
> > 
> > 
> > _______________________________________________
> > kontinuity-dev-public mailing list
> > kontinuity-dev-public at redhat.com
> > https://www.redhat.com/mailman/listinfo/kontinuity-dev-public
> > 
> 
>