[Container-tools] docker container based installers for Atomic / OpenShift V3 / GKE et al (was Re: Container App Spec and Tooling

Thu Apr 30 09:43:50 UTC 2015

I think having a docker container based installer of kubernetes apps or OpenShift templates is going to be very useful.

e.g. right now fabric8 creates a large number of apps; we’d like an easy way to install them (or a subset of them) on any kubernetes environment (e.g. Atomic / OpenShift / GKE) then have OpenShift specific extensions (routes and templates) so that when they are installed on OpenShift they can take advantage of the extensions (routes etc).

Its pretty trivial to take an OS template and expand it purely client side in the installer then turn it into normal kubernetes stuff; so am even tempted to just say everything’s an OpenShift Template (with some things not having any parameters) - then if not using OpenShift we’d expand the templates on the fly first before applying them?

Right now we’ve been using a bash script to install fabric8; but a more generic tool for installing M apps out of a possible N apps (with template parameterising) is going to be very useful I think.

The general idea of putting all the kubernetes / openshift json/yaml on disk some place; adding some extra ‘compose’ metadata on top (e.g. to group things into larger chunks) and then having reusable tools to install them sounds great.

Am sure lots of other parts of RHT are gonna be making kubernetes based apps with independent release cycles from the OpenShift project which may want to be consumed on any version of kubernetes / Atomic / OpenShift / GKE etc. (e.g. the cloud enablement folks are making apps for EAP and Fuse; the Management.Next folks already have apps for hawkular / cassandra / monitoring; we’ve loads of apps in fabric8 and then customers will have the same need for their apps).

So I like the idea of having a standard ‘installer’ docker container base we can add kubernetes & OpenShift metadata files into some folder and get an easy way to distribute and install things; either from the command line or via a nice little web app.

BTW we’d be happy to hack on a little HTML5 console for letting the user choose what apps to install and to create a little form to customise etc too. We’ve already a little UI for choosing apps to run in hawtio; so we’ve lots of the work done already.

Then the installation process would either use vanilla kubernetes REST APIs; optionally use OpenShift REST extensions and any extra Atomic stuff etc.

One key thing the installer needs to do is install all services first before any other resource (to fix any cyclical dependency issues; you need to pre-define the services to get them bound to IP addresses in kubernetes before starting any pods).

Another thing we’d find really handy is automatically creating OpenShift routes for all services given a configurable domain. e.g. service “foo” gets a route created at “foo.cheese.something.com <http://foo.cheese.something.com/>” if the DOMAIN=“cheese.something.com <http://cheese.something.com/>”. It doesn’t make sense to release OpenShift Routes metadata for each service; since they really just depend on the domain being used. (Though we maybe need to add annotations to services to customise the generation of them if a user wishes to add TLS termination stuff or maybe use a different host name other than ${serviceName}.${DOMAIN).

Another thing we’ve found handy with our current bash based installer is being able to run it purely in ‘download docker images first’ mode; together with an option to pull the latest images first before installing.

(More details on the bash script here:)
http://fabric8.io/guide/openShiftDocker.html#run-the-start-script <http://fabric8.io/guide/openShiftDocker.html#run-the-start-script>

Has work started on the installer? We kinda need one soon for fabric8; to more easily install it on OpenShift & on Atomic so we could maybe start hacking on the installer together? Is there an issue tracker we can start adding issues/ideas to?

> On 14 Apr 2015, at 15:14, Christoph Görn <goern at redhat.com> wrote:
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hey James,
> just a few hrs ago Aaron merged a big bunch of changes I made to the
> Container Application Specification. So please have a look at
> https://github.com/aweiteka/containerapp-spec/tree/master/spec/0.0.1-alp
> ha
> 
> We have had a look at fabric8 app zip and tried to consider the
> problem space you are working in, so any feedback is very welcome!
> 
> The idea in the end of you mail is pretty much the behavior we (Vasek)
> will implement in the Atomiapp Base Image we hope to have ready shortly.
> 
> Please keep in mind that we are going to also deliver ISV applications
> as a container (using docker container image format), that container
> shall be deployable on a variety of platforms (openshift,
> atomicplatform, rhel7+docker).
> 
> 	//G
> 
> On 04/13/2015 04:57 PM, Carl Trieloff wrote:
>> 
>> 
>> some good info.
>> 
>> 
>> -------- Forwarded Message -------- Subject: 	Re: Container App
>> Spec and Tooling Date: 	Mon, 13 Apr 2015 14:55:27 +0100 From:
>> James Strachan <jstracha at redhat.com> To: 	Aaron Weitekamp
>> <aweiteka at redhat.com> CC: 	cctrieloff at redhat.com, John Mark Walker
>> <jowalker at redhat.com>, Marek Goldmann <mgoldman at redhat.com>, James
>> Rawlings <jrawling at redhat.com>, Kevin Conner <kconner at redhat.com>,
>> Carl Trieloff <ctrielof at redhat.com>, Ian McLeod
>> <imcleod at redhat.com>
>> 
>> 
>> 
>>> On 13 Apr 2015, at 13:53, Aaron Weitekamp <aweiteka at redhat.com>
>>> wrote: ----- Original Message -----
>>>> 
>>>>> On 10 Apr 2015, at 19:22, Carl Trieloff
>>>>> <cctrieloff at redhat.com> wrote:
>>>>> 
>>>>> On 04/10/2015 01:24 PM, James Strachan wrote:
>>>>>> sounds good; I can’t make Tuesday but can do wednesday
>>>>>> though.
>>>>>> 
>>>>>> FWIW on the Java side of the house we’re using a maven
>>>>>> plugin which generates the kubernetes metadata for an
>>>>>> application (WAR, executable jar, Spring Boot, Camel Boot,
>>>>>> micro service, OSGi etc); it creates the Kubernetes Service
>>>>>> & Replication Controller etc: 
>>>>>> http://fabric8.io/guide/mavenPlugin.html#generating-the-json
>>>>> 
>>>>>> 
>>>>> 
>>>>> I would if we could also generate the container-app spec
>>>>> instance from the maven plugin?
>>>> 
>>>> we always welcome PRs :)
>>>> 
>>>> I don’t yet grok the value proposition for the Java person
>>>> who’s got a generated kubernetes JSON and an OpenShift Template
>>>> - what extra value does a generated container-app spec instance
>>>> add? Mind you I confess to not really grokking yet the
>>>> difference between a container-app spec yet and the existing
>>>> Kubernetes JSON/YAML & OpenShift Template file.
>>> 
>>> The confusion is reasonable given the early stage of this and our
>>> struggle to communicate it concisely. The problem we are
>>> addressing is how an ISV or an internal dev-ops shop packages and
>>> distributes an entire application with all of its dependencies
>>> and configuration options so that it can be run reliably.
>> 
>> We tried this out in fabric8 using an ‘app zip’ as a single binary
>> distro that can be run with 1 click on the fabric8 console etc 
>> http://fabric8.io/guide/appzip.html
>> 
>> e.g. here’s all the app zips for the standard apps in fabric8 right
>> now: 
>> http://central.maven.org/maven2/io/fabric8/jube/images/fabric8/apps/2.
> 0.41/
>> 
>> 
> http://central.maven.org/maven2/io/fabric8/jube/images/fabric8/apps/2.0.
> 41/apps-2.0.41-app.zip
>> 
>> and app zip is just a zip file of one or more folders of apps;
>> which each app is a folder with kube json, icon, readme docs etc.
>> Though we don’t put the docker images in the app zip; we assume
>> they are out of bounds via docker registries to make use of
>> docker’s layering &  caching.
>> 
>> The fabric8 console then lets you drag and drop them from your
>> desktop to the console or vice versa making it really easy to
>> release them or share them etc. 
>> http://fabric8.io/guide/console.html#library
>> 
>> 
>> Though we’re working on how to unify this with the OpenShift
>> templates; so that the maven plugin that generates the kube
>> json/app zip can just embed all the metadata inside an OpenShift
>> template as well. See this issue for more detail: 
>> https://github.com/openshift/origin/issues/1629
>> 
>> the remaining issue is how to install a group of templates
>> atomically (where each template can then be run separately); as
>> Clayton comments on the above issue, this could be a CLI command
>> with a wildcard or directory: 
>> https://github.com/openshift/origin/issues/1629#issuecomment-90982029
>> 
>> 
>> 
>>> We want to deal with containers, not configuration files. So any
>>> implementation of the spec results in a container, not a file or
>>> set of files that need to be managed. An ISV could hand you this
>>> container and it is all you need to run their application.
>> 
>> when you say container above do you mean it in the docker / LXC
>> sense or as in a collection of things in some kind of packaging? If
>> the former then its confusing ;) as kubernetes already deals with
>> JSON/YAML and separate docker images in a docker registry. A single
>> kube json / OpenShift template can map to many services and docker
>> containers.  If its the latter then going forward maybe use the
>> word “package” or some other word than container ;)
>> 
>> 
>>> The OpenShift template could be enough to describe a given
>>> application. We're taking those concepts and making the
>>> templating easier to develop, encourage composite or chaining of
>>> components and provide a standard for passing these things around
>>> (meta packaging).
>> 
>> OK. FWIW its pretty straightforward to take a number of OpenShift
>> templates and generate a new OpenShift template composing them
>> together (just concatenate the ‘items’ array in JSON); the main
>> complexity is the parameter expansion; do you assume each parameter
>> in each template is a separate value; or do you try to share
>> parameter names/values across templates in the composite (which
>> probably requires user configuration to differentiate).
>> 
>> Incidentally I’d like to see the Template extension from OpenShift
>> to be pushed up to Kubernetes too so its available in all
>> Kubernetes environments as its a powerful idea.
>> 
>> Looking forward to the meeting as we found manually creating
>> kubernetes json / OpenShift templates to be surprisingly error
>> prone (hence the maven plugin). Am sure we could use more tools to
>> make them easier to create/maintain.
>> 
>> 
>> One idea thats on my TODO list to experiment with was using
>> CoffeeScript as a DSL for generating kubernetes json / yaml /
>> OpenShift Templates as it already looks like YAML but with
>> functions and variable expansion; so its easy to reuse, for
>> example, labels or objects of env vars etc. Kinda like the Saas /
>> SCCS idea from CSS; adding variables/functions to a DSL helps make
>> more DRY metadata files.
>> 
>> e.g. a k8s DSL file could be like:
>> 
>> #/bin/k8sdsl
>> 
>> labels = x: "123" bar: "abc"
>> 
>> return service id: "James" selector: labels port: 3000
>> 
>> 
>> which then has a set of functions like ‘service’ to generate the
>> service json which are implicitly included by the k8sdsl script:
>> 
>> # default api version... apiVersion = "v1beta3"
>> 
>> # function to return a kubernetes Service service = (x) => 
>> x.apiVersion = apiVersion x.kind  = "Service" return x
>> 
>> .. ditto for all other k8s and openshift resource...
>> 
>> 
>> James ------- Red Hat
>> 
>> Twitter: @jstrachan Email: jstracha at redhat.com Blog:
>> http://macstrac.blogspot.com/
>> 
>> hawtio: http://hawt.io/ fabric8: http://fabric8.io/
>> 
>> Open Source Integration
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________ Container-tools
>> mailing list Container-tools at redhat.com 
>> https://www.redhat.com/mailman/listinfo/container-tools
>> 
> 
> - -- 
> Principal Software Engineer - Systems Design & Engineering
> Mobile: +49 171 2801345
> 
> Follow Us: https://twitter.com/RedHatRefArch
> Plus Us: https://plus.google.com/u/0/b/114152126783830728030/
> Like Us: https://www.facebook.com/rhrefarch
> 
> Red Hat GmbH, http://www.de.redhat.com/ Sitz: Grasbrunn,
> Handelsregister: Amtsgericht München, HRB 153243
> Geschäftsführer: Charles Cachera, Michael Cunningham, Michael O'Neill,
> Charles Peters
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
> 
> iQIcBAEBAgAGBQJVLSDcAAoJEKn71953Oyo0+A8P/iJ9jpRinUS8UcTxPMKTQGtp
> 1bpZfCkweKn6FDBpydjHAmql3Z/wTJIvW4ZGVfHmfk6XyskVMqyACH6cQ5KNuYQv
> znFpS0srvHNWnrsjP2uHH3BDtlp84ffJi5Q/d+Uqr4vBMfVUAAVB87EWoStM9S+6
> g9zv3pEUb3pQfRZnQtN/dHCinePSnv+WSo/eVyXeJwAwO6VpFHe9/cm2gJkGUP2t
> XIm4jTU1ZFgBpE9IqJZeYmM+rtKcinGljGXJBtW/ty1JUk3rsiKpP5m0hCG66MtV
> V54ZYcIKjvyhAk/3X80E6hUtWhcyaOqy6Ie7G+vfejfIis60l1tusssvCKu/3P+5
> l0fyILY3At/3g7oRQl9wCMKqDqWo2Y7EHRHraXrJ07j7smE69DULiIZAS/vreQvy
> pW0tcoRi04zXpKuFAtQwF6mwHcpqgH8j+WpOZOlCnAjvyM1oOKU8E+YChT7Ick5r
> gQe4UMqXfcZ8VP9VsLsNjG4GGoYh8SORWpWNLqLB0JBLjjRY6qv25upWqCKQF7nZ
> f/JcIWW7KvCPB9Wvh4t+J9G7zLIdNWwqrndrPaA5gZVQMFBsohXn+eo80+9ytlYI
> ev24z+xKUw1JlLjsMwO6zUeEC+yibpU/j6HBODILjLwJZS1ikPqDh2FtqXolcBie
> VzFRhfzT91QS2MZIEsAf
> =HopR
> -----END PGP SIGNATURE-----

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/container-tools/attachments/20150430/03b952fd/attachment.htm>