[Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib

[Jan Cholasta]

Dne 26.10.2011 16:39, Martin Kosek napsal(a):
> On Wed, 2011-10-26 at 15:52 +0200, Jan Cholasta wrote:
>> Dne 26.10.2011 15:41, Martin Kosek napsal(a):
>>> On Wed, 2011-10-26 at 11:39 +0200, Jan Cholasta wrote:
>>>> Dne 25.10.2011 22:30, Rob Crittenden napsal(a):
>>>>> Ondrej Hamada wrote:
>>>>>> On 10/25/2011 04:01 PM, Martin Kosek wrote:
>>>>>>> On Tue, 2011-10-25 at 15:29 +0200, Ondrej Hamada wrote:
>>>>>>>> https://fedorahosted.org/freeipa/ticket/1336
>>>>>>>>
>>>>>>>> Lazy initialization of ipalib plugins is used under all contexts, not
>>>>>>>> only when context = cli. Every loaded plugin is pre-finalized - a flag
>>>>>>>> is set, which means, that the plugin needs to be finalized. Then every
>>>>>>>> call of plugin's __gettattr__ checks the flag and finalizes the plugin
>>>>>>>> if necessary. The code was implemented by jcholast. Time reduction of
>>>>>>>> commands execution is quite markable:
>>>>>>>>
>>>>>>>> patch [s] | normal [s] | command
>>>>>>>> -----------------------------------------------------------------------
>>>>>>>> 1.468 | 2.287 | ipa user-add jsmith --firt=john
>>>>>>>> --last=smith
>>>>>>>> 1.658 | 2.235 | ipa user-del jsmith
>>>>>>>> 1.624 | 2.204 | ipa dnsrecord-find example.com
>>>>>>>>
>>>>>>> Thanks for submitting the patch. Ondra, just please provide the patch in
>>>>>>> proper format (exported via command `git format-patch -M -C --patience
>>>>>>> --full-index -1' which I sent you earlier).
>>>>>>>
>>>>>>> Martin
>>>>>>>
>>>>>>>
>>>>>> Sorry, correct version attached
>>>>>
>>>>> Wow, this is very impressive, great job Ondra and Honza!
>>>>>
>>>>> Martin, ACK from me but I'd like a second opinion. The patch is very
>>>>> straightforward and clean, just want to make sure we're not missing a
>>>>> corner case.
>>>>>
>>>>> I ran the self-tests and didn't see any problem there.
>>>>>
>>>>> Before pushing please add the ticket # to the commit.
>>>>>
>>>>> rob
>>>>>
>>>>
>>>> I've just remembered that special methods aren't looked up through
>>>> __getattribute__ (see the note at
>>>> http://docs.python.org/reference/datamodel.html#more-attribute-access-for-new-style-classes).
>>>> That might possibly cause problems in Command and subclasses, because it
>>>> uses __call__.
>>>>
>>>> Honza
>>>>
>>>
>>> This should be investigated. All our unit tests are OK though.
>>>
>>> I am also thinking if we shouldn't do this optimization on CLI side only
>>> as the original ticket suggests. As server side loads and finalizes all
>>> plugins just once at httpd start, performance should be OK. Plus, we
>>> would know right after the start that we are ready and there is no
>>> problem with finalizing any component.
>>>
>>> Martin
>>>
>>
>> As we discussed earlier - are you sure it works that way?
>
> Yes.
>
>>   I did a few
>> Django-based applications before and I'm pretty sure all mod_wsgi does
>> is caching the bytecode, which is run from the beginning for each
>> request, like what CLI does.
>
> I did few Django application myself too. Its all about how mod_wsgi is
> configured. You may want to learn more about WSGIDaemonProcess and
> WSGIImportScript directives.
>
> In IPA, we have API already prepared and finalized for every incoming
> request.
>
>>
>> If it does work the way you described, the worst thing that could happen
>> is that the plugins would be finalized the first time they are used and
>> then stay initialized until the server terminates.
>>
>> Honza
>>
>
> I know, I was just being conservative. As this optimization has no
> effect on server performance I considered it a little bit safer to load
> everything at start and be sure we are OK. But its not that hard
> requirement.
>
> Martin
>

OK, thanks, just wanted to be sure.

Honza

-- 
Jan Cholasta