[Container-tools] Docker developer tooling

[Brian (bex) Exelbierd]

On 07/24/2015 10:27 AM, Max Rydahl Andersen wrote:
> sounds good.
>
> btw. one little detail that I keep forgetting is that it would be good
> if the
> ip is static/stable.
>
> i.e. that the ip of the cdk stays the same between start/stops at least.

The IP will be localhost so stability of hte IP shouldn't be a problem.

However, while I have read that the port is stable from during 
start/stops, I don't have documentation that demonstrates that is reliable.

> I think that is already happening but just one of those things one keep
> forgetting that is
> pretty important as otherwise any setup you have will need to change the
> mentioned ip...

Each invocation of the script should return accurate information.

regards,

bex


>
> /max
>
>> On 07/23/2015 04:58 PM, Carl Trieloff wrote:
>> > On 07/23/2015 05:25 AM, Max Rydahl Andersen wrote:
>> >> TL;DR:
>> >>   Eclipse tooling currently just relies on same info as docker cli.
>> CDK
>> >> needs to allow connection from the host and provide a way to get the
>> >> connection information easily, e.g. "$(cdk docker-env)"
>>
>> I believe we should do this similarly to the way boot2docker does it.
>> 1) Expose the port for the docker daemon
>> 2) Generate certificates and share them
>> 3) Set the environment variables
>>
>> The challenges is:
>>
>> It appears the only way to expose a port from Vagrant is via a
>> portforward from the host. Vagrant supports port collision detection
>> and auto correction, however it doesn't have a programatic way to get
>> to the new port information.  It appears this is going to require a
>> provider level set of code to fix.  We should be able to write a
>> plugin, or leverage some parts of existing ones to figure this out.
>>
>> I believe that once we have this, we can write a simple script to
>> generate the environment variables required,
>>
>> I am working down this road now.
>>
>> >> and it should
>> >> be possible to easily mount local filesystems.
>>
>> I haven't explored this yet, but my first-blush take is that it is
>> going to require virtualbox additions for the virtualbox cdk boxes.
>> The libvirt boxes should be fine, as they can use nfs.
>>
>> regards,
>>
>> bex
>>
>> >>
>> >> The long version:
>> >>
>> >> General challenge
>> >> -----------------
>> >>
>> >>  From developer experience side we must all realise that the OS
>> >> users/developers are running on is *not* a Linux based OS, and if it
>> >> is Linux it most likely is not RHEL. They will often go to production
>> >> on RHEL but for sure during development at least 80% of our
>> >> customers/users are on Windows and the rest on a mixture of Linux and
>> >> OS X distributions.
>> >>
>> >> Furthermore since we want to target enterprise developers most of them
>> >> prefer to use a UI or if they do use the CLI they prefer being able to
>> >> use the tools that runs on their own host rather than through a
>> >> tunnelled shell. This does not mean we should not expose shells nor
>> >> cli's - just means that we *cannot* ignore the UI and direct access
>> >> options.
>> >>
>> >> In our eclipse tooling we are doing what we can to provide a good UI
>> >> experience while still coexisting with the CLI tools; preferably
>> >> thought without *relying* on users to install the cli tools. As an
>> >> example our Openshift v2 tools required no custom cli setup *at all*
>> >> on any OS to use, it just worked out of the box. Super easy to setup
>> >> and get started with.
>> >>
>> >> We want to have similar experience in as many usecases as possible for
>> >> our new generation of CDK/docker/openshift tooling story too.
>> >>
>> >> Connection
>> >> ----------
>> >>
>> >> The docker tooling in Eclipse is just accessing docker as if it was
>> >> any other docker client. It supports accessing docker over tcp and
>> >> over raw sockets. The raw sockets especially is nice in the sense that
>> >> it on Linux hosts allow the setup to "just work" on default Linux
>> >> docker installs.
>> >>
>> >> We want to get an equal simple experience on systems where you do not
>> >> have raw sockets access, such as Windows, OS X or remote docker
>> >> setups. Thus on such systems we run a little script to try and fetch
>> >> the following environment variables: `DOCKER_HOST`, `DOCKER_CERT_PATH`
>> >> and `DOCKER_TLS_VERIFY` and if found we use that as the default setup.
>> >>
>> >> This means that if the user has configured docker in his setup it will
>> >> again "just work".
>> >>
>> >> The nice thing about this mechanism is that if you use `boot2docker`
>> >> you can just add the following line to your setup: `$(boot2docker
>> >> shellinit)` or if you use the newer `docker-machine` and have a docker
>> >> setup called `dev` you can add `"$(docker-machine env dev)"`
>> >>
>> >> This works because these commands spits out something like:
>> >>
>> >> ```
>> >> export DOCKER_HOST=tcp://192.168.59.103:2376
>> >> export DOCKER_CERT_PATH=/Users/max/.boot2docker/certs/boot2docker-vm
>> >> export DOCKER_TLS_VERIFY=1
>> >> export DOCKER_MACHINE_NAME="dev"
>> >> ```
>> >>
>> >> And this is exactly what docker cli uses and so does the eclipse
>> tooling.
>> >>
>> >> Thus in short, anything that makes it easy to use CDK from the *host*
>> >> cli will be beneficial for our Eclipse usage too.
>> >>
>> >> For now we read environment variables for the setup (which will work
>> >> with "anything"), but going forward we also would like to
>> >> add plugins to detect running docker instances available to the user
>> >> allowing docker for eclipse to find docker setup via boot2docker,
>> >> docker-machine, openshift connections and in this context a local
>> >> running CDK.
>> >>
>> >> ![](cid:DAA600B7-99E8-4CF5-9C17-A4C832103B76 at redhat.com
>> >> "PastedImage.png")
>> >>
>> >> Volumes
>> >> -------
>> >>
>> >> Above is just about the actual connection to Docker, beyond this is
>> >> also the importance of efficiently being able to mount volumes from
>> >> user host into the docker instance.
>> >>
>> >> Again, here `boot2docker` and `docker-machine` makes this work `out of
>> >> the box` by allowing mount of host user filesystems.
>> >>
>> >> This is what we utilise in blogs like:
>> >> http://tools.jboss.org/blog/docker_tooling_eclipse_mars.html and
>> >>
>> http://tools.jboss.org/blog/2015-03-02-getting-started-with-docker-and-wildfly.html
>>
>> >>
>> >> Thus beyond being able to connect to docker inside the CDK we also
>> >> need to be able to have support for volume mount.
>> >>
>> >> I believe vagrant has *some* support for mounting volumes but last
>> >> time I tried this that was more a sync rather than an actual mount
>> >> meaning changes was slow and somewhat unreliable compared to what I've
>> >> seen with boot2docker and docker-machine.
>> >>
>> >> I'm hoping you guys can tell me I'm wrong about this and vagrant can
>> >> be made to work just as well as boot2docker and docker-machine
>> >>
>> >> Docker Actions
>> >> --------------
>> >>
>> >> Currently what I've been able to get info about the CDK the docker
>> >> connection is only accessible from within the vagrant host by running
>> >> `vagrant ssh` and there was no (easy nor uniform) way to mount volumes
>> >> from my local system to docker or any other part of the CDK.
>> >>
>> >> Thus for the CDK I see the first task to be that the CDK by default
>> >> make access to Docker possible from the host running the CDK
>> >> and then provide a command to dump the environment settings, for
>> example:
>> >>
>> >> ```
>> >> $ cdk docker-env
>> >> export DOCKER_HOST=tcp://192.168.59.103:2376
>> >> export DOCKER_CERT_PATH=/Users/max/.cdk/certs/docker-vm
>> >> export DOCKER_TLS_VERIFY=1
>> >> export CDK_DOCKER_NAME="redhat-cdk"
>> >> ```
>> >>
>> >> Then users can add `$(cdk docker-env)` to their setup and Eclipse
>> >> tooling will just work.
>> >>
>> >> Note the presence of access to the host stored DOCKER_CERT_PATH. These
>> >> cannot be paths *within* the vagrant image since neither a docker cli
>> >> or eclipse can reach those.
>> >>
>> >> And then we need to find a way on how volume mounts will work.
>> >>
>> >> Bonus OpenShift
>> >> ---------------
>> >>
>> >> Our OpenShift tooling in eclipse does not (yet) use environment
>> >> variables but just needs a URL to the OpenShift 3 console and it will
>> >> figure out how to connect. Thus the OpenShift instance in CDK should
>> >> also be default exposed and accessible to the running host.
>> >>
>> >> Maybe even have a `cdk openshift-env` to get the details.
>> >>
>> >>
>> >> The End.
>> >>
>> >> I hope that gives some insights - if any questions do not hesitate to
>> >> ping me on this mail thread or on irc (nick: maxandersen).
>> >>
>> >> /max
>> >> http://about.me/maxandersen
>> >
>
>
> /max
> http://about.me/maxandersen