[EnMasse] Adapting the EnMasse deployment

Tue Jun 20 15:45:39 UTC 2017

Hi Ulf,

thanks, this clarifies it.
And an interesting solution, dealing with the missing templating feature in Kubernetes (just had a look at the corresponding k8s github issue).

Best regards

Carsten Lohmann

(INST/ECS4)
Bosch Software Innovations GmbH | Schöneberger Ufer 89-91 | 10785 Berlin | GERMANY | www.bosch-si.com<http://www.bosch-si.com>
Tel. +49 30 726112-130 | Fax +49 30 726112-100 | carsten.lohmann at bosch-si.com<mailto:carsten.lohmann at bosch-si.com>

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Geschäftsführung: Dr.-Ing. Rainer Kallenbach, Michael Hahn

Von: Ulf Lilleengen [mailto:ulilleen at redhat.com]
Gesendet: Montag, 19. Juni 2017 13:33
An: Lohmann Carsten (INST/ECS4) <Carsten.Lohmann at bosch-si.com>
Cc: enmasse at redhat.com
Betreff: Re: [EnMasse] Adapting the EnMasse deployment

Hi Carsten,

In EnMasse, there are currently 4 types of addresses: 'anycast', 'multicast', 'queue', 'topic'. The first two are 'direct' address that only makes use of the dispatch router. Whenever you create an address with 'queue' or 'topic' semantics, you get a broker. The way this is done is that whenever you create a queue (store_and_forward=true, multicast=false, flavor='vanilla-queue' for instance), then the address-controller will create the Deployment for a broker to handle that queue. If an address with the same group exists, it will reuse that deployment and the queue on the broker will be created. See https://github.com/EnMasseProject/enmasse/blob/master/documentation/getting-started/openshift.md#configuring-addresses for more on address configuration.

My slides from RivieraDev may help in explaining how brokers are created today: https://www.slideshare.net/UlfLilleengen/rivieradev-75928765

The conceptual model around how addresses are defined is subject to change, and a design draft will be published on this.

The address-controller is the component that applies the JSON in those config maps. It uses the fabric8 kubernetes client, which has a capability of processing OpenShift templates locally even when running on Kubernetes. This essentially allows us to use OpenShift templates for the 'instance infrastructure (router ++)' (only 1 instance in the single-tenant case) and the 'broker infrastructure' (multiple of these depending on the address config) on Kubernetes.

An OpenShift template is basically a list of Kubernetes resources that can be processed (parameter substitution) and then applied (creates the processed resources). When running on OpenShift, the address controller processes the templates 'remotely' rather than from the resources in the config map.

When running on Kubernetes, the 'enmasse-template-config' configmap is mounted within the address-controller pod, and i.e. whenever you create a queue, it will process one of the template files within that ConfigMap and substitute parameters like the ADDRESS etc (which originally comes from the user input).

Ulf

On Mon, Jun 19, 2017 at 1:04 PM, Lohmann Carsten (INST/ECS4) <Carsten.Lohmann at bosch-si.com<mailto:Carsten.Lohmann at bosch-si.com>> wrote:
Good, thank you for the info.

One further difference in the Hono qdrouter config is the inclusion of the artemis broker.

While I see an artemis image being referenced in the nested JSONs of the Kubernetes enmasse.yaml,
it seems there is no such container being deployed after applying enmasse.yaml.
Is inclusion of the broker in Kubernetes currently not enabled/supported?
One thing I'm still trying to grasp is the way the Kubernetes resources defined in the ConfigMap with the JSONs
actually get applied/deployed.
Do you have pointers here on the corresponding Kubernetes mechanisms used for that?

Best regards

 Carsten Lohmann

(INST/ECS4)
Bosch Software Innovations GmbH | Schöneberger Ufer 89-91 | 10785 Berlin | GERMANY | www.bosch-si.com<http://www.bosch-si.com>
Tel. +49 30 726112-130<tel:%2B49%2030%20726112-130> | Fax +49 30 726112-100<tel:%2B49%2030%20726112-100> | carsten.lohmann at bosch-si.com<mailto:carsten.lohmann at bosch-si.com>

Sitz: Berlin, Registergericht: Amtsgericht Charlottenburg; HRB 148411 B
Geschäftsführung: Dr.-Ing. Rainer Kallenbach, Michael Hahn

-----Ursprüngliche Nachricht-----
Von: Ulf Lilleengen [mailto:ulilleen at redhat.com<mailto:ulilleen at redhat.com>]
Gesendet: Freitag, 16. Juni 2017 14:30
An: Lohmann Carsten (INST/ECS4) <Carsten.Lohmann at bosch-si.com<mailto:Carsten.Lohmann at bosch-si.com>>; enmasse at redhat.com<mailto:enmasse at redhat.com>
Betreff: Re: [EnMasse] Adapting the EnMasse deployment

On 16. juni 2017 12:44, Ulf Lilleengen wrote:
> On 16. juni 2017 12:08, Lohmann Carsten (INST/ECS4) wrote:
>> Hi Ulf,
>>
>>>  Out of curiosity, what is it that you wish to modify in this config?
>>
>> We want to use a config similar to the one used in Hono:
>>
>> https://github.com/eclipse/hono/blob/master/dispatchrouter/qpid/qdrou
>> terd-with-broker.json
>>
>>  > I.e. with our sslProfile / certificates and vhost definitions.
>>
>
> One thing to look out for there is that the enmasse router config is
> created dynamically from a static fixed template + configuration from
> the router agent (address config for instance).
>
> To make it work properly in EnMasse, you have to merge that config
> with the static enmasse router config:
>
> https://github.com/EnMasseProject/dockerfiles/blob/master/qdrouterd/qd
> routerd.conf.template
>
>

Just to elaborate on this part: Eventually we hope to provide a way in EnMasse to do this without overriding the router config. For certs, you can edit the certificates used by the router by creating/editing the secret 'certs-$namespace' where $namespace is the namespace where you deployed EnMasse to, which will be used for external connections.

We intend to improve the certificate management in the near future in combination with keycloak integration.

How to add vhost definitions is something that needs more discussion, but we're working on a backlog so this is useful input.

--
Ulf
_______________________________________________
enmasse mailing list
enmasse at redhat.com<mailto:enmasse at redhat.com>
https://www.redhat.com/mailman/listinfo/enmasse

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/enmasse/attachments/20170620/e6f20a69/attachment.htm>