[EnMasse] Upcoming REST API changes

[Ulf Lilleengen]

On 05/08/2018 02:27 PM, Gordon Sim wrote:
> On 08/05/18 11:47, Ulf Lilleengen wrote:
>>
>> tir. 8. mai 2018 kl. 03:23 skrev Gordon Sim <gsim at redhat.com 
>> <mailto:gsim at redhat.com>>:
>>
>>     On 08/05/18 11:13, Ulf Lilleengen wrote:
>>      > One of my goals with this work is that we in the future could
>>     migrate
>>      > all components today working on configmaps to use this API instead
>>      > (through the kubernetes master). This would allow us to change
>>     how we
>>      > persist the data from configmaps to something more fitting.  My
>>     worry is
>>      > that if we put all addresses into the address space definition,
>>     we would
>>      > end up facing issues in standard-controller and agent due to 
>> status
>>      > updates needing to fetch/query a lot of data when one of the
>>     addresses
>>      > change.
>>
>>     At present they need to do that anyway, as there is no reliable 
>> way of
>>     tracking changes (especially deletions) except by periodically
>>     retrieving the whole list.
>>
>>
>> The periodic resync  is less frequent than the status checks which is 
>> what typically trigger the watches a lot when configuring many 
>> addresses. If we put addresses into then address space definition, 
>> then all addresses is read on every update to any address, whereas 
>> today only a single address is read when the status is changed.
> 
> As you say, if you are reading at the granularity of the address space, 
> then a small update involves extra data transfer. This is the same as 
> happens on the resync.
> 
> Again, as you rightly point out, the frequency at which this happens is 
> relevant.
> 
> The granularity and frequency are often related though.
> 
> A single logical update that adds (or deletes or modifies) thousands of 
> addresses results in thousands of notifications when watching at the 
> granularity of the address.
> 
> If instead you work at the granularity of the address space, for updates 
> and reads/watches, my feeling is that the frequency of changes will be 
> much less and therefore that the extra data transferred is not 
> prohibitive especially in comparison to the resync that happens anyway.
> 
> I.e. though at present during an update we get a storm of notifications 
> with a much higher frequency than the resync interval, my instinct is 
> that in general logical updates will be probably even less frequent than 
> the resync.

Agreed with this logic, it might not be an issue then. So just thinking 
about the resource then, it could be something like:

apiVersion: enmasse.io/v1
kind: AddressSpace
metadata:
   name: myspace
spec:
   type: standard
   plan: small-standard
   addresses:
     - address: myqueue
       type: queue
       plan: small-queue
     - address: mytopic
       type: topic
       plan: small-topic
status:
   isReady: true
   addresses:
     - address: myqueue
       phase: Configuring
       isReady: false
       messages:
         - "Autolink not present on router"
         - "Address not present on router"
     - address: mytopic
       phase: Active
       isReady: true

Making status a subresource, we can then also assign permissions so that 
only the internal controllers may update the status whereas the user 
only get read permissions (same as for pods).

I think doing this would be a great simplification from a user POV, so I 
really hope it can work.

As a first step, I can modify the API server to consume this format and 
convert to/from the configmaps we use today.

Wdyt? Are there any tests we should/can do to ensure that this approach 
will actually work? Any other concerns?

Best regards,

Ulf