[EnMasse] AddressController performance / ConfigMap lookups

[Jakub Scholz]

There used to be a limit of 1MB per ConfigMap AFAIK (which is driven by
etcd default record size, so CRD would have similar limit). If that is
still valid, do you plan to split the addresses into several ConfigMaps?
Because 1MB doesn't give you much space for 10000+ addresses.

Regards
Jakub

On Tue, Mar 13, 2018 at 10:47 AM, Ulf Lilleengen <lulf at redhat.com> wrote:

>
>
> On 03/13/2018 10:40 AM, Gordon Sim wrote:
>
>> On 13/03/18 09:16, Ulf Lilleengen wrote:
>>
>>> If one address status needs to be changed, the controller then have to
>>> read/write 1000 addresses worth of data, so I wouldn't say that it is more
>>> manageable. If the problem is that the kubernetes master is overloaded by
>>> enmasse use, this could even make it worse.
>>>
>>
>> It is certainly speculative, but often smaller numbers of slightly larger
>> data items are easier to manage and cause less load. While updating status
>> would indeed cause some unneeded data transfer, retrieval of the addresses
>> (which I would argue is by far the more common operation, assuming the
>> addresses change relatively infrequently on a running system) would involve
>> far fewer calls. Likewise for a bulk update of addresses (10,000 PUTs will
>> generally cause more load that 10, even if the latter have more data in
>> them).
>>
>>
> I think we already read addresses in batches by reading a ConfigMapList
> (though maybe that translates to 10000 reads from etcd, I'm not sure if it
> supports it natively).
>
> A PUT with AddressList currently generates 10000 configmap PUTs. However,
> I think that could be improved by moving to custom resources as well, as
> you get 'native' support in the kubernetes API for doing a list PUT.
>
> In any case though, I would suggest we start with some actual experiments
>> on pushing the scale wrt the number of addresses and find out where the
>> actual bottlenecks are and figure out how to alleviate them.
>>
>
> Fully agreed, numbers don't lie (usually :))
>
> Best regards,
>
> Ulf
>
>
> _______________________________________________
> enmasse mailing list
> enmasse at redhat.com
> https://www.redhat.com/mailman/listinfo/enmasse
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/enmasse/attachments/20180313/4ca0feb2/attachment.htm>