[almighty] Ensuring Production-like Environments

[Max Rydahl Andersen]

On 2 Nov 2016, at 21:59, Pete Muir wrote:

> Adding Kev, Rob and Diogenes as they are also working on these sorts
> of issues. Also Clayton as he and I have discussed this many times and
> have a conceptual understanding of how to resolve at least... ;-)

One thing to be aware of if you are not is that for the existing 
developer tooling (wether doing from eclipse, maven or inhand via cli's) 
the docker images from us have rsync "friendly" setups. It has labels 
set to let tooling know about where to publish content and flags to turn 
on/off hotreload and debugging.

This allows for somewhat shared images without adding extra burden or 
overhead into the production images.

Does not solve all - but at least can be used for fairly efficient 
incremental updates.

But i'm not sure any of Che's dev/run modes can currently support 
incremental updates ?
>
> On 2 November 2016 at 05:00, Tomas Nozicka <tnozicka at redhat.com> 
> wrote:
>> On Út, 2016-11-01 at 14:24 -0400, Andrew Lee Rubinger wrote:
>>> The general premise we need to get to is:
>>> every environment as close to production as possible."
>>>
>>> With containers we're now closer than we've been before to doing 
>>> development, testing, builds, and pre-production deployments in a 
>>> production-like environment.
>> yes, it should be a standard since we have containers :)
>>
>>>
>>> The naive approach we could take looks like a single base image to 
>>> be used across the board for online IDE development, testing, 
>>> executing integration tests in the pipeline, deploying to 
>>> pre-production, and production.  For instance, we may associate an 
>>> EAP codebase with a centos-eap base image that's used throughout 
>>> Eclipse Che and the various OpenShift deployments for testing and 
>>> poking around during development.  The differences between this and 
>>> production may be limited to externalizable sysprops, environment 
>>> variables, persistent volume mounts, etc.
>> Having a single image for dev, build, pipeline and prodution would 
>> result
>> in bloated image for prod which we probably don't want. And e.g. 
>> pipelines
>> right now require special image (with jenkins slave packages)
>> so that would not work there. (But pipelines can use some common 
>> image for build.)
>
> Agreed. This was an early problem with OpenShift s2i that I believe is
> being fixed.
>
> We need builder images that create runtime images. I know that we have
> patterns from OpenShift and the MW cloud enablement team that we can
> copy here.
>
>>
>>> But the problem has nuance.  I'd like us to arrive at a proposal for 
>>> how we associate a "runtime" with a codebase.  Some questions to 
>>> address:
>>>
>>> 1) A Git repository (Almighty "codebase") may have N target runtimes 
>>> contained within.
>> yes, and these runtimes should be part of ConfigMaps
>>   http://kubernetes.io/docs/user-guide/configmap/
>> and other types for this task like secrets, which are separate yaml
>> files and the application definition stays the same across all
>> environments
>
> This requires some patterns. Here I think that the kompose work is
> particuarly important, as well as the Che support for importing
> compose based definitions.
>
> How do we plan to support this in the build and test phases? That I'm
> not sure about.
>
>>
>>> 2) Che "receipes" (images) require root or sudo access, plus other 
>>> tools[1].  Does that mean we have to make an "IDE" image that 
>>> extends from our base image?  We can't have the container (OS-level) 
>>> user executing our servers/frameworks running as root.
>> Any image that runs as root *won't* be allowed to run on OpenShift,
>> especially on Online! There is an option to allow it but inherently
>> this means a *security breach*, so don't expect it in any non test
>> installation!
>
> The root problem is specific to Che, and requires the Che to
> rearchitect/fix the parts of Che that require root. They are already
> working on this and Mario did a call today to present where they are
> at - he can post the recording. This is part of a broader challenge
> they have to make Che run on an unmodified (i.e. secured) OpenShift.
> They are making good progress, they have workspaces running in Pods,
> but otherwise still require the Docker API and root for quite a bit.
>
> The tools thing is part of a wider problem that requires some
> significant R&D. As an example, this problem exists for Jenkins slaves
> as well - ideally they need to be able to execute a java agent on any
> image you can get from docker hub. And not just on a Java based image,
> you need to do this on a Node image or a Go image as well. We
> basically need a way to "inject" a Java based tool in to any image and
> have it run without problems.

Injection of "anything" is whats really needed - but I reckon if we can 
inject a java based tool then we can inject anything else to.

For me this is not just a need for development vs runtime phase - 
similar needs are present for being able to debug and introspect a 
runtime docker image no matter what state it is in.

But for some reason profiling tools etc. are considered okey to include, 
but not build tools into docker images. Haven't really understood that 
mindset - both are equally bad security and overhead wise :)

>>
>>> Or that the image used to build extends from the image to run?  For
>> instance, EAP applications need the JDK to run, but additionally need
>> Maven to build.
>> I would say this may work for JDK/Maven and many others, but ain't
>> generally true. I could thing of an image that needs some components
>> for runtime that are not needed for build.
>
> I think that builder images are a better pattern, and certainly one
> adopted by others.

Anyone seen a setup with openshift where builder images don't result in 
massive increase in build since the machine that is building don't have 
the previous builds cached for the docker images ?

Similar issue with the mvn setup for java, except instead of re-fetching 
tons of small dependencies it now needs to fetch several layers of 
prebuilt images.


/max
http://about.me/maxandersen