[EnMasse] Upcoming REST API changes

[Ulf Lilleengen]

Hi all,

Thanks for the good discussion and feedback in this thread. I have 
limited the scope of this PR to these user visible changes:

* Enable namespacing of address spaces
* Move REST API path for addresses so that they are nested under address 
spaces
* Make address spaces available as a custom resource

I think the above changes are things all agree on is a step in the right 
direction. Further changes like embedding addresses within the address 
space definition or provide addresses as a custom resource are both 
potential extensions (that by no means are mutual exclusive) that we 
should get a more detailed proposal sorted out for.

For now, I will merge the PR before the next release.

Best regards,

Ulf

On 05/08/2018 11:20 PM, Ulf Lilleengen wrote:
> On 05/08/2018 10:54 PM, Ulf Lilleengen wrote:
>> On 05/08/2018 02:27 PM, Gordon Sim wrote:
>>> On 08/05/18 11:47, Ulf Lilleengen wrote:
>>>>
>>>> tir. 8. mai 2018 kl. 03:23 skrev Gordon Sim <gsim at redhat.com 
>>>> <mailto:gsim at redhat.com>>:
>>>>
>>>>     On 08/05/18 11:13, Ulf Lilleengen wrote:
>>>>      > One of my goals with this work is that we in the future could
>>>>     migrate
>>>>      > all components today working on configmaps to use this API 
>>>> instead
>>>>      > (through the kubernetes master). This would allow us to change
>>>>     how we
>>>>      > persist the data from configmaps to something more fitting.  My
>>>>     worry is
>>>>      > that if we put all addresses into the address space definition,
>>>>     we would
>>>>      > end up facing issues in standard-controller and agent due to 
>>>> status
>>>>      > updates needing to fetch/query a lot of data when one of the
>>>>     addresses
>>>>      > change.
>>>>
>>>>     At present they need to do that anyway, as there is no reliable 
>>>> way of
>>>>     tracking changes (especially deletions) except by periodically
>>>>     retrieving the whole list.
>>>>
>>>>
>>>> The periodic resync  is less frequent than the status checks which 
>>>> is what typically trigger the watches a lot when configuring many 
>>>> addresses. If we put addresses into then address space definition, 
>>>> then all addresses is read on every update to any address, whereas 
>>>> today only a single address is read when the status is changed.
>>>
>>> As you say, if you are reading at the granularity of the address 
>>> space, then a small update involves extra data transfer. This is the 
>>> same as happens on the resync.
>>>
>>> Again, as you rightly point out, the frequency at which this happens 
>>> is relevant.
>>>
>>> The granularity and frequency are often related though.
>>>
>>> A single logical update that adds (or deletes or modifies) thousands 
>>> of addresses results in thousands of notifications when watching at 
>>> the granularity of the address.
>>>
>>> If instead you work at the granularity of the address space, for 
>>> updates and reads/watches, my feeling is that the frequency of 
>>> changes will be much less and therefore that the extra data 
>>> transferred is not prohibitive especially in comparison to the resync 
>>> that happens anyway.
>>>
>>> I.e. though at present during an update we get a storm of 
>>> notifications with a much higher frequency than the resync interval, 
>>> my instinct is that in general logical updates will be probably even 
>>> less frequent than the resync.
>>
>> Agreed with this logic, it might not be an issue then. So just 
>> thinking about the resource then, it could be something like:
>>
>> apiVersion: enmasse.io/v1
>> kind: AddressSpace
>> metadata:
>>    name: myspace
>> spec:
>>    type: standard
>>    plan: small-standard
>>    addresses:
>>      - address: myqueue
>>        type: queue
>>        plan: small-queue
>>      - address: mytopic
>>        type: topic
>>        plan: small-topic
>> status:
>>    isReady: true
>>    addresses:
>>      - address: myqueue
>>        phase: Configuring
>>        isReady: false
>>        messages:
>>          - "Autolink not present on router"
>>          - "Address not present on router"
>>      - address: mytopic
>>        phase: Active
>>        isReady: true
>>
>> Making status a subresource, we can then also assign permissions so 
>> that only the internal controllers may update the status whereas the 
>> user only get read permissions (same as for pods).
>>
>> I think doing this would be a great simplification from a user POV, so 
>> I really hope it can work.
>>
>> As a first step, I can modify the API server to consume this format 
>> and convert to/from the configmaps we use today.
>>
>> Wdyt? Are there any tests we should/can do to ensure that this 
>> approach will actually work? Any other concerns?
>>
> 
> Responding to myself here :) Combining the addresses in the address 
> space requires some changes to the address controller as well if we want 
> to avoid rewriting standard-controller, agent etc.
> 
> The namespace where the address configmaps are stored is not created 
> until the address controller processes the address space. The API server 
> cannot create these configmaps unless the namespace already exists. To 
> get around this, we could try to have the API server create the 
> namespace when handling the http request, instead of the 
> address-controller creating it.
> 
> Another alternative is to just dive into the process of rewriting the 
> standard-controller, agent etc. to use the API which would move us 
> towards the goal of pluggable storage faster, but at the risk of 
> changing many things at once.
> 

-- 
Ulf