[Freeipa-devel] Session design document

[John Dennis]

On 12/01/2011 09:55 PM, Simo Sorce wrote:
> On Wed, 2011-11-30 at 17:33 -0500, John Dennis wrote:
>> Comments? Suggestions?
>>
> Sorry for the late reply.
>
> First of all, excellent write-up John, it is very comprehensive and lays
> down things very clearly.

Thank you.

> I agree that using ipa:ipa for memcached and wsgi would be a better
> proposition for us. Although we need to explore how this would affect
> credential caches created by mod_auth_kerb and our ability to use them,
> which is crucial*.
>
> You say that object sized for the stuff we will store in memcached
> should be limited. What is a reasonable size for those objects ?
> I was thinking we may want to store the krb ccaches in memcached in
> order to be able to keep them around. The reason I ask is that Krb
> ccaches can become quite big if PACs are attached to tickets although
> they are normally quite small.

I had to do a little digging to get the answer.

The maximum size of a single entry is 1 MB. That includes some metadata 
bound to the item for internal use, but it isn't a lot, so technically a 
single item's max size is just under one megabyte. This limit can be 
increased but the memcached developers do not recommend it (I believe 
because it reduces memory usage efficiency due to page hits).

By default the memcached daemon initializes with 64 MB of total memory 
the vast majority of which is allocated to item storage and a minor 
amount allocated to bookkeeping. The total memory size is easily 
adjusted by a command line parameter when the daemon initializes.

In addition the Python library supports compression using zlib. You 
specify a minimum compression length. The value is converted into it's 
storage form (python objects other than integers and strings are 
"pickled") and if the length of the value in it's final storage form is 
greater than the minimum compression length zib is invoked to compress 
it. This way zlib compression is avoided for small items (or compression 
can be completely disabled). Thus we have the option to play around with 
compressing our entries if we discover we have some need to. I think 
initially we would disable compression sacrificing memory consumption in 
favor of performance.

My guess is we're not likely to be bumping up against the 1 MB per item 
threshold (nor would it be smart to anywhere be close to that). I think 
I recalled you mentioning that PAC data would max out around 16 KB. So I 
don't see the limit as being something we realistically need to worry 
about (or at least I hope not :-)

One nice feature of memcached is you can query the statistics from a 
live daemon at any time. While we're tuning we might need to do this.

> Aside for these minor details I totally agree with the direction you are
> proposing and I can't wait to see it implemented :)
>
>
> Simo.
>
>
> *In the long term we may even decide to stop using mod_auth_kerb and do
> our own handling if this simplifies things, but I guess we will need an
> interim transition period in any case, because we can't depend on too
> many changes to be done at once as a dependency to introduce sessions.
>


-- 
John Dennis <jdennis at redhat.com>

Looking to carve out IT costs?
www.redhat.com/carveoutcosts/