[EnMasse] External Kubernetes services

[Ulf Lilleengen]

Hi Carsten,

A fix has been merged to master 
(https://github.com/EnMasseProject/enmasse/pull/658) and will be part of 
0.16.0.

I've filed an issue for making endpoints support more configuration 
options (https://github.com/EnMasseProject/enmasse/issues/661).

Best regards,

Ulf

On 15. des. 2017 14:38, Lohmann Carsten (INST/ECS4) wrote:
> Hi Ulf,
> 
>> In 'service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "certName"'
>> what does certName point to? A secret or some other resource?
> That is an Amazon Resource Name ("arn:aws:..").
> 
> Best regards
>   Carsten Lohmann
> 
> 
>> -----Ursprüngliche Nachricht-----
>> Von: Ulf Lilleengen [mailto:lulf at redhat.com]
>> Gesendet: Freitag, 15. Dezember 2017 13:35
>> An: Lohmann Carsten (INST/ECS4) <Carsten.Lohmann at bosch-si.com>;
>> enmasse at redhat.com
>> Betreff: Re: AW: [EnMasse] External Kubernetes services
>>
>> Hi Carsten,
>>
>> The endpoints only indicate what is being exposed, and does not change what is
>> deployed. I will fix so that you can do that which should give you the freedom to
>> roll your own.
>>
>> Regarding the aws service definition below, I will write up a proposal for improving
>> this.
>>
>> In 'service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "certName"'
>> what does certName point to? A secret or some other resource?
>>
>> Best regards,
>>
>> Ulf
>>
>> On 15. des. 2017 13:19, Lohmann Carsten (INST/ECS4) wrote:
>>> Hi Ulf,
>>>
>>> here is an example with the added modifications of our "messaging-external"
>> service:
>>> ---------------------
>>> - apiVersion: v1
>>>     kind: Service
>>>     metadata:
>>>       labels:
>>>         app: enmasse
>>>       annotations:
>>>         dns.alpha.kubernetes.io/external: our-messaging-domain-name.
>>>         service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "certName"
>>>         service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "5671"
>>>       name: messaging-external
>>>     spec:
>>>       ports:
>>>       - name: amqps
>>>         port: 5671
>>>         protocol: TCP
>>>         # TLS gets terminated at the LoadBalancer, target port here 5672
>>>         targetPort: 5672
>>>       selector:
>>>         capability: router
>>>       type: LoadBalancer
>>>       loadBalancerSourceRanges: ..ip list..
>>> ---------------------
>>> So it's about added annotations, different target port, added
>> "loadBalancerSourceRanges".
>>>
>>> Regarding using an empty endpoint list of the address space:
>>> If the endpoints only correspond to K8s services, that's OK. I was only
>> wondering if an omitted endpoint also leads to other corresponding resources
>> (e.g. pods) not being created.
>>> See https://github.com/EnMasseProject/enmasse/issues/544.
>>>
>>>
>>> Best regards
>>>
>>>    Carsten Lohmann
>>>
>>>> -----Ursprüngliche Nachricht-----
>>>> Von: Ulf Lilleengen [mailto:lulf at redhat.com]
>>>> Gesendet: Donnerstag, 14. Dezember 2017 17:41
>>>> An: Lohmann Carsten (INST/ECS4) <Carsten.Lohmann at bosch-si.com>;
>>>> enmasse at redhat.com
>>>> Betreff: Re: [EnMasse] External Kubernetes services
>>>>
>>>> Hi Carsten,
>>>>
>>>> The intention is that you can override this when creating the address
>>>> space by specifying an empty endpoint list like this:
>>>>
>>>> {
>>>>      "kind": "AddressSpace",
>>>>      "spec": {
>>>>         "endpoints": []
>>>>      }
>>>> }
>>>>
>>>> Unfortunately, it appears that this does not work as intended. I have
>>>> created an issue to address this[1], but I'd like to know if that is
>>>> sufficient for your use case before starting on a fix.
>>>>
>>>> Any details you can share on what properties are required for you to
>>>> be able to configure your loadbalancer using the address space
>>>> resource rather than having to create them explicitly would be
>>>> valuable. If you have an example LoadBalancer service definition, I
>>>> can try to propose a way to configure endpoints in sufficient detail that will give
>> you the LoadBalancer service you wish.
>>>>
>>>> [1] https://github.com/EnMasseProject/enmasse/issues/648
>>>>
>>>> Best regards,
>>>>
>>>> Ulf
>>>>
>>>> On 14. des. 2017 16:57, Lohmann Carsten (INST/ECS4) wrote:
>>>>> Hi,
>>>>>
>>>>> in EnMasse 0.15.0 there has been this change:
>>>>>
>>>>> "Replace use of Ingress with K8S LoadBalancer Service"
>>>>>
>>>>> https://github.com/EnMasseProject/enmasse/commit/f2680732530e37ff5fa
>>>>> 85
>>>>> c02ff58617a002e2640
>>>>>
>>>>> Having the K8S LoadBalancer Services be created automatically by the
>>>>> Address Controller proves to be rather inflexible in our use case.
>>>>>
>>>>> Before EnMasse 0.15.0, we have used an adapted "external-lb.yaml"
>>>>> file with added annotations
>>>>>
>>>>> - concerning definition of DNS records for the created LoadBalancer,
>>>>> definition of LoadBalancer SSL certificate/port, etc.
>>>>>
>>>>> Now to have these annotations, we must either adapt or
>>>>> delete/re-create the Services.
>>>>>
>>>>> And since in the AWS setup there is the creation of dependent AWS
>>>>> LoadBalancer entries, there is quite some overhead involved in this.
>>>>>
>>>>> Would it be feasible to add a flag to skip automatic Service creation?
>>>>> Or changing the code-wise creation to one based on a yaml file that
>>>>> can be adapted?
>>>>>
>>>>> Best regards
>>>>>
>>>>> *Carsten Lohmann
>>>>> *
>>>>> (INST/ECS4)
>>>>> Bosch Software Innovations GmbH | Ullsteinstr. 128 | 12109 Berlin |
>>>>> GERMANY| www.bosch-si.com <http://www.bosch-si.com>
>>>>>
>>>>> Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB
>>>>> 148411 B
>>>>> Aufsichtsratsvorsitzender: Dr.-Ing. Thorsten Lücke; Geschäftsführung:
>>>>> Dr.-Ing. Rainer Kallenbach, Michael Hahn
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> enmasse mailing list
>>>>> enmasse at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/enmasse
>>>>>