[Container-tools] Atomic Developer Bundle and OpenShift

Tue Nov 3 09:21:56 UTC 2015

On 11/03/2015 05:09 AM, Praveen Kumar wrote:
> On Mon, Nov 2, 2015 at 11:04 PM, Langdon White <langdon at redhat.com> wrote:
>> Hi list,
>>
>> I think we have a number of moving parts for getting OpenShift integrated in
>> to the ADB vagrant box and I am nervous we don't have all the aspects
>> assigned to anyone (on either the c-t team or the OpenShift team). I am also
>> concerned that I might be missing some aspects. Please weigh in if you own
>> one of these pieces and/or if you think we are missing any.
>>
>> * a method for docker images to be pre-loaded on to the vagrant boxes: As
>> you probably agree, we would really like the v-up experience of the ADB to
>> be as quick and painless as possible. One of things that will make that
>> possible is to "pre-install" the docker images for OpenShift, AtomicApp,
>> v2c, etc. However, the build tooling (koji) does not allow a build to access
>> the general internet. As a result, "docker pull" is not an option (at least
>> from docker-hub). We have a couple options here:
>>    * stand up a docker registry in the build environment that the builds can
>> access: While this seems like a good idea, the timeline to make this happen
>> is probably on the order of months not days
>>    * auto-rpm-ify the docker images: Build the images in koji, use koji to
>> rpm-ify the binary images, pull the rpms as per normal, extract the rpm and
>> inject them in to the docker-images storage. Likely, this is the most viable
>> solution. However, it may run in to problems with docker-registry-v2
>> (doesn't support import at this time).
>> Is anyone owning testing and resolving this issue?
>>
>> * OpenShift needs dns to allow a user to access their applications: For
>> OpenShift to give a good user experience, it needs to manage some wildcard
>> domain. In other words, when a user sets up an application, they need to
>> give it a name and they access the application from their host web browser
>> at something like "myCoolApp.myADB.lcl". OpenShift uses host-headers to
>> route the browser to the correct app. However, this means, if OpenShift is
>> running in a VM, the host machine needs to know to route *.myADB.lcl to the
>> VM and then to OpenShift. As the VM will come up on an (likely) unknowable
>> IP, we planned to use vagrant-landrush, a plugin for vagrant that manages a
>> DNS server for this type of use case. Currently, this plugin still has some
>> problems on windows and has never been tested in this exact use case. Is
>> someone working on:
>>    1) testing that this setup will actually work with OpenShift (even on mac
>> or linux where, i believe, v-landrush is known to work)
>>    2) looking in to the issues on windows?
>>
>> What landrush, loosely, does:
>> on start of the vagrant vm; vagrant launches the box; vagrant calls
>> landrush; landrush looks at the ip of the vagrant-vm; inserts *.myADB.lcl ->
>> vagrant-vm-ip;
>>
>> full example:
>> web browser gets user request for mycoolnewwebsite.myADB.lcl; browser goes
>> to host resolution (where that is depends on OS), in there it finds
>> *.myADB.lcl-> vagrant-vm-ip; browser then navigates to that ip; OpenShift in
>> the VM listens on that address:80, OpenShift looks at host-header
>> (mycoolnewwebsite.myADB.lcl) and maps it to the correct running website in
>> OpenShift
>>
>> * allow for k8s + docker to work independently of OpenShift: In the plans
>> for ADB we wanted to allow a developer to use k8s+docker directly as well as
>> OpenShift. However, this is not quite as easy as it seems as the k8s version
>> on CentOS and the k8s version in OpenShift are not the same. As a result,
>> even if they are installed separately (see installation bullets elsewhere),
>> they need to be listening on different IP bindings to allow them to listen
>> on the same port. Does someone own testing and ensuring the setup of these
>> conflicting services?
>
> I was working with k8s + openshift work independently part and tried
> different method [0], [1] to isolate those services to avoid conflicts
> and did discuss it with our technical meeting. The issue is still not
> resolved and I had a discussion with openshift dev member (Maciej
> Szulik) who said that is not a good idea and in ideal situation one
> service should be running at any given point (k8s or openshift) [2]. I
> would love to look it again if we have some way/suggestion to go
> about. I will also check if bind will work as per our requirement.

My $0.02 in the topic, to clear out the situation and save some time.
OpenShift is built on top of k8s, so when running OpenShift you actually
run k8s with OpenShift. See for example this piece of start code [1] and
you'll see how we start k8s controllers, on which OS heavily relays on.
That's why running both OpenShift and k8s is not possible without
too much of a hassle. Additionally there's one more point to remember,
when two k8s master will be running they will start fighting for the
pods they've started and nodes they control, which will result in more
mess you would want to have.
Given above arguments it's perfectly understandable to show and explain
you should have one or the other running at any given point in time.
Besides when running OS you still have full access to k8s api in
any way you can have when running plain k8s, iow. both kubectl and
REST API work, this can be seen by looking again at OS source code [2]
or logs from OS start.
I hope that explains more, and if you have more questions don't hesitate
to ask.

Maciej

[1] 
https://github.com/openshift/origin/blob/master/pkg/cmd/server/start/start_master.go#L528-L557
[2] 
https://github.com/openshift/origin/blob/master/pkg/cmd/server/kubernetes/master.go#L50-L56

>
> [0] http://post-office.corp.redhat.com/archives/aos-devel/2015-October/msg00936.html
> [1] http://blog.scottlowe.org/2013/09/04/introducing-linux-network-namespaces/
> [2] http://fpaste.org/286287/23068144/
>
>>
>>
>> Is that it?
>>
>> langdon
>
>
>