[Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib

[Jan Cholasta]

Dne 26.10.2011 15:41, Martin Kosek napsal(a):
> On Wed, 2011-10-26 at 11:39 +0200, Jan Cholasta wrote:
>> Dne 25.10.2011 22:30, Rob Crittenden napsal(a):
>>> Ondrej Hamada wrote:
>>>> On 10/25/2011 04:01 PM, Martin Kosek wrote:
>>>>> On Tue, 2011-10-25 at 15:29 +0200, Ondrej Hamada wrote:
>>>>>> https://fedorahosted.org/freeipa/ticket/1336
>>>>>>
>>>>>> Lazy initialization of ipalib plugins is used under all contexts, not
>>>>>> only when context = cli. Every loaded plugin is pre-finalized - a flag
>>>>>> is set, which means, that the plugin needs to be finalized. Then every
>>>>>> call of plugin's __gettattr__ checks the flag and finalizes the plugin
>>>>>> if necessary. The code was implemented by jcholast. Time reduction of
>>>>>> commands execution is quite markable:
>>>>>>
>>>>>> patch [s] | normal [s] | command
>>>>>> -----------------------------------------------------------------------
>>>>>> 1.468 | 2.287 | ipa user-add jsmith --firt=john
>>>>>> --last=smith
>>>>>> 1.658 | 2.235 | ipa user-del jsmith
>>>>>> 1.624 | 2.204 | ipa dnsrecord-find example.com
>>>>>>
>>>>> Thanks for submitting the patch. Ondra, just please provide the patch in
>>>>> proper format (exported via command `git format-patch -M -C --patience
>>>>> --full-index -1' which I sent you earlier).
>>>>>
>>>>> Martin
>>>>>
>>>>>
>>>> Sorry, correct version attached
>>>
>>> Wow, this is very impressive, great job Ondra and Honza!
>>>
>>> Martin, ACK from me but I'd like a second opinion. The patch is very
>>> straightforward and clean, just want to make sure we're not missing a
>>> corner case.
>>>
>>> I ran the self-tests and didn't see any problem there.
>>>
>>> Before pushing please add the ticket # to the commit.
>>>
>>> rob
>>>
>>
>> I've just remembered that special methods aren't looked up through
>> __getattribute__ (see the note at
>> http://docs.python.org/reference/datamodel.html#more-attribute-access-for-new-style-classes).
>> That might possibly cause problems in Command and subclasses, because it
>> uses __call__.
>>
>> Honza
>>
>
> This should be investigated. All our unit tests are OK though.
>
> I am also thinking if we shouldn't do this optimization on CLI side only
> as the original ticket suggests. As server side loads and finalizes all
> plugins just once at httpd start, performance should be OK. Plus, we
> would know right after the start that we are ready and there is no
> problem with finalizing any component.
>
> Martin
>

As we discussed earlier - are you sure it works that way? I did a few 
Django-based applications before and I'm pretty sure all mod_wsgi does 
is caching the bytecode, which is run from the beginning for each 
request, like what CLI does.

If it does work the way you described, the worst thing that could happen 
is that the plugins would be finalized the first time they are used and 
then stay initialized until the server terminates.

Honza

-- 
Jan Cholasta