[Freeipa-users] 2.20 dirsrv memory usage

[Petr Vobornik]

On 07/18/2012 08:59 PM, Stephen Ingram wrote:
> On Wed, Jul 18, 2012 at 6:45 AM, Petr Vobornik <pvoborni at redhat.com> wrote:
>> On 07/17/2012 11:43 PM, Stephen Ingram wrote:
>>
>> 8><------
>>
>>
>>>>>
>>>>> I'm beginning to think this is just the Web UI itself instead of 389
>>>>> although it is really difficult to tell. I've poured over the debug
>>>>> logs and didn't see anything that caused me concern.
>>>>>
>>>>> It's certainly usable, but I just got really spoiled by the
>>>>> unbelievable quickness of 2.1.3. When your release notes indicate it
>>>>> should be faster, what are you comparing it to? Maybe this only
>>>>> happens with upgraded instances and not fresh installs.
>>>>
>>>>
>>>>
>>>> It is always possible something didn't get upgraded properly but I've
>>>> done
>>>> 2.1.3 -> 2.2.0 upgrades and haven't seen this. When we say something is
>>>> faster we're always referring to the previous version (or versions).
>>>
>>>
>>> Maybe I was just lucky with 2.1.3. On a first load it might take some
>>> time to load the "frame" as I call it. But the data would load almost
>>> instantaneously from there (certainly no more than 1 s) as you moved
>>> from page to page. Here, even if I return to the same page, the system
>>> acts as if the data is begin fetched for the very first time as it is
>>> no faster than the first load. Maybe that is significant to the
>>> problem?
>>
>>
>> I think the culprit is Web UI paging capabilities introduced in 2.2. With
>> lot of users, responses might grow in size. You can check their size and
>> duration in browser developers tools. I suggest chrome/chromium - press F12
>> and choose 'network' tab.
>>
>> This new feature can't be disabled in configuration. To test if the slowdown
>> is done by paging you can (at own risk) replace line
>> /usr/share/ipa/ui/facet.js:538
>>
>> that.pagination = spec.pagination === undefined ? true : spec.pagination;
>>
>> with:
>>
>> that.pagination = false;
>>
>> Note: It will break some other parts of the UI - so for testing only.
>
> I've made the substitution in the code (was line 507 for me-do I have
> a different version?).

I was looking at the top of FreeIPA 2.2 branch. RHEL version differs a 
bit. It shouldn't matter in this case though.

> Looking at the time chart in Chrome I see that
> the bulk of the time is for /ipa/session waiting. Would "waiting" mean
> waiting for the directory server or memcached?

Basically all the stuff, which is needed for processing of the request. 
The pipeline is something like (I don't want to go into details): httpd 
-> mod_wsgi -> python -> memcache, dir server request .... and back.

 From what I see I think the problem is not on Web UI side. Most of the 
time is waiting for server response.

I initially thought the problem lies in a large number of users (1000+). 
But in other post you mentioned that it is under 100. Hence this new 
paging feature shouldn't be a big problem.


>
> Here's a portion of the initial load of the Users page:

The first 3 requests are inital load of web UI (not including .js files 
and such) - you can see they are the same in both cases. I don't see a 
login request so session is already established.

> json/ipa/session POST 200 Success application/json jquery.js:7365
> Script 33.94KB 33.10KB 1.57s 1.47s 96ms (1.37s waiting)
> json/ipa/session POST 200 Success application/json jquery.js:7365
> Script 568.09KB 564.36KB 3.92s 2.95s 963ms (2.85s waiting)
> json/ipa/session POST 200 Success application/json jquery.js:7365
> Script 556.94KB 553.40KB 3.78s 2.94s 836ms (2.83s waiting)

This one is user load:
> json/ipa/session POST 200 Success application/json jquery.js:7365
> Script 46.93KB 46.38KB 1.87s 1.71s (1.60s waiting)
>
> Now, with the pagination turned back on:
> json/ipa/session POST 200 Success application/json jquery.js:7365
> Script 33.94KB 33.10KB 1.58s 1.48s 100ms (1.38s waiting)
> json/ipa/session POST 200 Success application/json jquery.js:7365
> Script 568.09KB 564.36KB 4.05s 3.09s 964ms (2.98s waiting)
> json/ipa/session POST 200 Success application/json jquery.js:7365
> Script 556.94KB 553.40KB 3.84s 2.99s 855ms (2.88s waiting)

Here you can see a change. With pagination turned on it should be two 
request. One to get primary keys (logins) and second to get users. The 
latter is missing in this list. With a lot of users the first response 
grows. With low number of users it is in fact smaller than with 
pagination turned off.
> json/ipa/session POST 200 Success application/json jquery.js:7365
> Script 46.93KB 46.38KB 1.52s 1.51s (1.40s waiting)

As I mentioned you are missing one request so it will add aprox. 1.5s. 
The second request is kinda a slowdown from IPA 2.1.4 but the main issue 
is still the long server processing time.

>
> Steve
>


-- 
Petr Vobornik