[Container-tools] Atomic Developer Bundle and OpenShift

[Praveen Kumar]

On Tue, Nov 3, 2015 at 10:35 PM, Langdon White <langdon at redhat.com> wrote:
>
>
> On 11/03/2015 04:21 AM, Maciej Szulik wrote:
>>
>> My $0.02 in the topic, to clear out the situation and save some time.
>> OpenShift is built on top of k8s, so when running OpenShift you actually
>> run k8s with OpenShift. See for example this piece of start code [1] and
>> you'll see how we start k8s controllers, on which OS heavily relays on.
>> That's why running both OpenShift and k8s is not possible without
>> too much of a hassle. Additionally there's one more point to remember,
>> when two k8s master will be running they will start fighting for the
>> pods they've started and nodes they control, which will result in more
>> mess you would want to have.
>> Given above arguments it's perfectly understandable to show and explain
>> you should have one or the other running at any given point in time.
>> Besides when running OS you still have full access to k8s api in
>> any way you can have when running plain k8s, iow. both kubectl and
>> REST API work, this can be seen by looking again at OS source code [2]
>> or logs from OS start.
>> I hope that explains more, and if you have more questions don't hesitate
>> to ask.
>>
>> Maciej
>>
>>
>> [1]
>> https://github.com/openshift/origin/blob/master/pkg/cmd/server/start/start_master.go#L528-L557
>> [2]
>> https://github.com/openshift/origin/blob/master/pkg/cmd/server/kubernetes/master.go#L50-L56
>>
>
> So.. the problem is that the two versions are different. So, if you are
> gonna deploy in prod on RHEL/CentOS (or atomic), i believe you get the
> rhel/centos version of k8s. As a result, it is not a good test to use the
> OpenShift k8s. So, I think we all understand that OpenShift is using k8s,
> the problem is it is the wrong version. IIUC.

Yes, this is what our current issue is (version mismatch). Yesterday I
again had chat with Maciej and Michal about same. According to them in
future this will be taken care of by openshift team (same version of
k8s they are going to use with OS also). I also did some experiment[0]
around k8s api which openshift provide and there is some network issue
for service (I have to follow up with Maciej for same).

>
> I also understand it is painful to bind to different IPs and potential
> conflicts on docker, however, I think it is worth the experimentation to
> simplify the experience for the end user.

Yes it is painful but I got a response to try bind() function[1] with
different IP + services which I will check today and see if that
workout and let you folks know. k8s and etcd services have options to
run it at different port then default and when I used this option, I
was able to run k8s services along with openshift but then whatever
pods were running before on plain k8s  just crushed as soon as OS
service started. So even we try to map different service with
different IP to make sure port conflict not occur then also this issue
will bug us. (Maciej also faced same issue when I had discussion with
him and then suggested to use single service at given time)

>
> If the experiment fails, or proves to be too much effort, then, yes, I think
> turning them each "on and off" is probably the best answer.

Yes, this is our last option.

[0] http://fpaste.org/286696/60948414/
[1] http://linux.die.net/man/2/bind

-- 
Praveen Kumar
http://fedoraproject.org/wiki/User:Kumarpraveen
http://fedoraproject.org/
http://kumar-pravin.blogspot.com