[Container-tools] Atomic Developer Bundle and OpenShift

Wed Nov 4 13:31:18 UTC 2015

Ideally the version we have in RHEL should match the version we have in
OpenShift (at least on the API level).

On Wed, Nov 4, 2015 at 1:45 PM, Vaclav Pavlin <vpavlin at redhat.com> wrote:

>
>
> On Wed, Nov 4, 2015 at 1:39 PM, Maciej Szulik <maszulik at redhat.com> wrote:
>
>>
>>
>> On 11/04/2015 05:06 AM, Praveen Kumar wrote:
>>
>>> On Tue, Nov 3, 2015 at 10:35 PM, Langdon White <langdon at redhat.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On 11/03/2015 04:21 AM, Maciej Szulik wrote:
>>>>
>>>>>
>>>>> My $0.02 in the topic, to clear out the situation and save some time.
>>>>> OpenShift is built on top of k8s, so when running OpenShift you
>>>>> actually
>>>>> run k8s with OpenShift. See for example this piece of start code [1]
>>>>> and
>>>>> you'll see how we start k8s controllers, on which OS heavily relays on.
>>>>> That's why running both OpenShift and k8s is not possible without
>>>>> too much of a hassle. Additionally there's one more point to remember,
>>>>> when two k8s master will be running they will start fighting for the
>>>>> pods they've started and nodes they control, which will result in more
>>>>> mess you would want to have.
>>>>> Given above arguments it's perfectly understandable to show and explain
>>>>> you should have one or the other running at any given point in time.
>>>>> Besides when running OS you still have full access to k8s api in
>>>>> any way you can have when running plain k8s, iow. both kubectl and
>>>>> REST API work, this can be seen by looking again at OS source code [2]
>>>>> or logs from OS start.
>>>>> I hope that explains more, and if you have more questions don't
>>>>> hesitate
>>>>> to ask.
>>>>>
>>>>> Maciej
>>>>>
>>>>>
>>>>> [1]
>>>>>
>>>>> https://github.com/openshift/origin/blob/master/pkg/cmd/server/start/start_master.go#L528-L557
>>>>> [2]
>>>>>
>>>>> https://github.com/openshift/origin/blob/master/pkg/cmd/server/kubernetes/master.go#L50-L56
>>>>>
>>>>>
>>>> So.. the problem is that the two versions are different. So, if you are
>>>> gonna deploy in prod on RHEL/CentOS (or atomic), i believe you get the
>>>> rhel/centos version of k8s. As a result, it is not a good test to use
>>>> the
>>>> OpenShift k8s. So, I think we all understand that OpenShift is using
>>>> k8s,
>>>> the problem is it is the wrong version. IIUC.
>>>>
>>>
>> Langdon, can you please specify what you mean by OpenShift is using
>> wrong version of k8s? We're trying hard to follow upstream, our
>> current version is 20 days old [1] with quite a lot of cherry-picks
>> that are crucial for our 1.1 release soon to happen.
>>
> Hi Maciej,
>
> I think Langdon meant the difference between version of Kubes used in OS
> and shipped in RHEL Atomic (but that's just my rough guess:) )
>
> Cheers,
> Vašek
>
>>
>> [1]
>> https://github.com/kubernetes/kubernetes/commit/4c8e6f47ec23f390978e651232b375f5f9cde3c7
>>
>> Yes, this is what our current issue is (version mismatch). Yesterday I
>>> again had chat with Maciej and Michal about same. According to them in
>>> future this will be taken care of by openshift team (same version of
>>> k8s they are going to use with OS also). I also did some experiment[0]
>>> around k8s api which openshift provide and there is some network issue
>>> for service (I have to follow up with Maciej for same).
>>>
>>>
>>>> I also understand it is painful to bind to different IPs and potential
>>>> conflicts on docker, however, I think it is worth the experimentation to
>>>> simplify the experience for the end user.
>>>>
>>>
>>> Yes it is painful but I got a response to try bind() function[1] with
>>> different IP + services which I will check today and see if that
>>> workout and let you folks know. k8s and etcd services have options to
>>> run it at different port then default and when I used this option, I
>>> was able to run k8s services along with openshift but then whatever
>>> pods were running before on plain k8s  just crushed as soon as OS
>>> service started. So even we try to map different service with
>>>
>>
>> This is the case I was talking about. K8s is very aggressive when it
>> comes to managing docker containers. If you watch carefully every
>> container started by k8s has a `k8s_` prefix, with this in mind you
>> might imagine what happens when two k8s (the stadalone one and the one
>> OpenShift one) run, trying to take over all the containers with that
>> prefix. And containers is not the only part that k8s very aggressively
>> manages, others that I can think of will be networking which provides
>> access to the internal parts of the created infrastructure. The bind
>> problem Praveen is facing now is the first one on the long list of
>> problems, I imagine, you'll have to deal with.
>>
>> different IP to make sure port conflict not occur then also this issue
>>> will bug us. (Maciej also faced same issue when I had discussion with
>>> him and then suggested to use single service at given time)
>>>
>>
>> I'm in contact with Praveen about solving problems he's having to run
>> services on top of the internal OpenShift k8s, but most of them are
>> due to the security constraints OpenShift provides to restrict
>> access to the k8s infrastructure.
>>
>>
>>>> If the experiment fails, or proves to be too much effort, then, yes, I
>>>> think
>>>> turning them each "on and off" is probably the best answer.
>>>>
>>>
>>> Yes, this is our last option.
>>>
>>> [0] http://fpaste.org/286696/60948414/
>>> [1] http://linux.die.net/man/2/bind
>>>
>>>
>> _______________________________________________
>> Container-tools mailing list
>> Container-tools at redhat.com
>> https://www.redhat.com/mailman/listinfo/container-tools
>>
>
>
>
> --
> Architect - Senior Software Engineer
> Developer Experience
> Brno, Czech Republic
> Phone: +420 739 666 824
>
> _______________________________________________
> Container-tools mailing list
> Container-tools at redhat.com
> https://www.redhat.com/mailman/listinfo/container-tools
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/container-tools/attachments/20151104/e312bbef/attachment.htm>