[EnMasse] AddressController performance / ConfigMap lookups

[Ulf Lilleengen]

At present, 1 address = 1 configmap. The alternative under discussion is 
to do N addresses = 1 configmap, where N is not something we really know.

On 03/13/2018 11:08 AM, Jakub Scholz wrote:
> There used to be a limit of 1MB per ConfigMap AFAIK (which is driven by 
> etcd default record size, so CRD would have similar limit). If that is 
> still valid, do you plan to split the addresses into several ConfigMaps? 
> Because 1MB doesn't give you much space for 10000+ addresses.
> 
> Regards
> Jakub
> 
> On Tue, Mar 13, 2018 at 10:47 AM, Ulf Lilleengen <lulf at redhat.com 
> <mailto:lulf at redhat.com>> wrote:
> 
> 
> 
>     On 03/13/2018 10:40 AM, Gordon Sim wrote:
> 
>         On 13/03/18 09:16, Ulf Lilleengen wrote:
> 
>             If one address status needs to be changed, the controller
>             then have to read/write 1000 addresses worth of data, so I
>             wouldn't say that it is more manageable. If the problem is
>             that the kubernetes master is overloaded by enmasse use,
>             this could even make it worse.
> 
> 
>         It is certainly speculative, but often smaller numbers of
>         slightly larger data items are easier to manage and cause less
>         load. While updating status would indeed cause some unneeded
>         data transfer, retrieval of the addresses (which I would argue
>         is by far the more common operation, assuming the addresses
>         change relatively infrequently on a running system) would
>         involve far fewer calls. Likewise for a bulk update of addresses
>         (10,000 PUTs will generally cause more load that 10, even if the
>         latter have more data in them).
> 
> 
>     I think we already read addresses in batches by reading a
>     ConfigMapList (though maybe that translates to 10000 reads from
>     etcd, I'm not sure if it supports it natively).
> 
>     A PUT with AddressList currently generates 10000 configmap PUTs.
>     However, I think that could be improved by moving to custom
>     resources as well, as you get 'native' support in the kubernetes API
>     for doing a list PUT.
> 
>         In any case though, I would suggest we start with some actual
>         experiments on pushing the scale wrt the number of addresses and
>         find out where the actual bottlenecks are and figure out how to
>         alleviate them.
> 
> 
>     Fully agreed, numbers don't lie (usually :))
> 
>     Best regards,
> 
>     Ulf
> 
> 
>     _______________________________________________
>     enmasse mailing list
>     enmasse at redhat.com <mailto:enmasse at redhat.com>
>     https://www.redhat.com/mailman/listinfo/enmasse
>     <https://www.redhat.com/mailman/listinfo/enmasse>
> 
> 
> 
> 
> _______________________________________________
> enmasse mailing list
> enmasse at redhat.com
> https://www.redhat.com/mailman/listinfo/enmasse
> 

-- 
Ulf