[Container-tools] Atomic Developer Bundle and OpenShift

Wed Nov 4 12:45:03 UTC 2015

On Wed, Nov 4, 2015 at 1:39 PM, Maciej Szulik <maszulik at redhat.com> wrote:

>
>
> On 11/04/2015 05:06 AM, Praveen Kumar wrote:
>
>> On Tue, Nov 3, 2015 at 10:35 PM, Langdon White <langdon at redhat.com>
>> wrote:
>>
>>>
>>>
>>> On 11/03/2015 04:21 AM, Maciej Szulik wrote:
>>>
>>>>
>>>> My $0.02 in the topic, to clear out the situation and save some time.
>>>> OpenShift is built on top of k8s, so when running OpenShift you actually
>>>> run k8s with OpenShift. See for example this piece of start code [1] and
>>>> you'll see how we start k8s controllers, on which OS heavily relays on.
>>>> That's why running both OpenShift and k8s is not possible without
>>>> too much of a hassle. Additionally there's one more point to remember,
>>>> when two k8s master will be running they will start fighting for the
>>>> pods they've started and nodes they control, which will result in more
>>>> mess you would want to have.
>>>> Given above arguments it's perfectly understandable to show and explain
>>>> you should have one or the other running at any given point in time.
>>>> Besides when running OS you still have full access to k8s api in
>>>> any way you can have when running plain k8s, iow. both kubectl and
>>>> REST API work, this can be seen by looking again at OS source code [2]
>>>> or logs from OS start.
>>>> I hope that explains more, and if you have more questions don't hesitate
>>>> to ask.
>>>>
>>>> Maciej
>>>>
>>>>
>>>> [1]
>>>>
>>>> https://github.com/openshift/origin/blob/master/pkg/cmd/server/start/start_master.go#L528-L557
>>>> [2]
>>>>
>>>> https://github.com/openshift/origin/blob/master/pkg/cmd/server/kubernetes/master.go#L50-L56
>>>>
>>>>
>>> So.. the problem is that the two versions are different. So, if you are
>>> gonna deploy in prod on RHEL/CentOS (or atomic), i believe you get the
>>> rhel/centos version of k8s. As a result, it is not a good test to use the
>>> OpenShift k8s. So, I think we all understand that OpenShift is using k8s,
>>> the problem is it is the wrong version. IIUC.
>>>
>>
> Langdon, can you please specify what you mean by OpenShift is using
> wrong version of k8s? We're trying hard to follow upstream, our
> current version is 20 days old [1] with quite a lot of cherry-picks
> that are crucial for our 1.1 release soon to happen.
>
Hi Maciej,

I think Langdon meant the difference between version of Kubes used in OS
and shipped in RHEL Atomic (but that's just my rough guess:) )

Cheers,
Vašek

>
> [1]
> https://github.com/kubernetes/kubernetes/commit/4c8e6f47ec23f390978e651232b375f5f9cde3c7
>
> Yes, this is what our current issue is (version mismatch). Yesterday I
>> again had chat with Maciej and Michal about same. According to them in
>> future this will be taken care of by openshift team (same version of
>> k8s they are going to use with OS also). I also did some experiment[0]
>> around k8s api which openshift provide and there is some network issue
>> for service (I have to follow up with Maciej for same).
>>
>>
>>> I also understand it is painful to bind to different IPs and potential
>>> conflicts on docker, however, I think it is worth the experimentation to
>>> simplify the experience for the end user.
>>>
>>
>> Yes it is painful but I got a response to try bind() function[1] with
>> different IP + services which I will check today and see if that
>> workout and let you folks know. k8s and etcd services have options to
>> run it at different port then default and when I used this option, I
>> was able to run k8s services along with openshift but then whatever
>> pods were running before on plain k8s  just crushed as soon as OS
>> service started. So even we try to map different service with
>>
>
> This is the case I was talking about. K8s is very aggressive when it
> comes to managing docker containers. If you watch carefully every
> container started by k8s has a `k8s_` prefix, with this in mind you
> might imagine what happens when two k8s (the stadalone one and the one
> OpenShift one) run, trying to take over all the containers with that
> prefix. And containers is not the only part that k8s very aggressively
> manages, others that I can think of will be networking which provides
> access to the internal parts of the created infrastructure. The bind
> problem Praveen is facing now is the first one on the long list of
> problems, I imagine, you'll have to deal with.
>
> different IP to make sure port conflict not occur then also this issue
>> will bug us. (Maciej also faced same issue when I had discussion with
>> him and then suggested to use single service at given time)
>>
>
> I'm in contact with Praveen about solving problems he's having to run
> services on top of the internal OpenShift k8s, but most of them are
> due to the security constraints OpenShift provides to restrict
> access to the k8s infrastructure.
>
>
>>> If the experiment fails, or proves to be too much effort, then, yes, I
>>> think
>>> turning them each "on and off" is probably the best answer.
>>>
>>
>> Yes, this is our last option.
>>
>> [0] http://fpaste.org/286696/60948414/
>> [1] http://linux.die.net/man/2/bind
>>
>>
> _______________________________________________
> Container-tools mailing list
> Container-tools at redhat.com
> https://www.redhat.com/mailman/listinfo/container-tools
>

-- 
Architect - Senior Software Engineer
Developer Experience
Brno, Czech Republic
Phone: +420 739 666 824
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/container-tools/attachments/20151104/6664a130/attachment.htm>