[Freeipa-devel] [PATCH] 1 Do lazy initializiation ipalib

[Jan Cholasta]

Dne 26.10.2011 16:50, Jan Cholasta napsal(a):
> Dne 26.10.2011 16:39, Martin Kosek napsal(a):
>> On Wed, 2011-10-26 at 15:52 +0200, Jan Cholasta wrote:
>>> Dne 26.10.2011 15:41, Martin Kosek napsal(a):
>>>> On Wed, 2011-10-26 at 11:39 +0200, Jan Cholasta wrote:
>>>>> Dne 25.10.2011 22:30, Rob Crittenden napsal(a):
>>>>>> Ondrej Hamada wrote:
>>>>>>> On 10/25/2011 04:01 PM, Martin Kosek wrote:
>>>>>>>> On Tue, 2011-10-25 at 15:29 +0200, Ondrej Hamada wrote:
>>>>>>>>> https://fedorahosted.org/freeipa/ticket/1336
>>>>>>>>>
>>>>>>>>> Lazy initialization of ipalib plugins is used under all
>>>>>>>>> contexts, not
>>>>>>>>> only when context = cli. Every loaded plugin is pre-finalized -
>>>>>>>>> a flag
>>>>>>>>> is set, which means, that the plugin needs to be finalized.
>>>>>>>>> Then every
>>>>>>>>> call of plugin's __gettattr__ checks the flag and finalizes the
>>>>>>>>> plugin
>>>>>>>>> if necessary. The code was implemented by jcholast. Time
>>>>>>>>> reduction of
>>>>>>>>> commands execution is quite markable:
>>>>>>>>>
>>>>>>>>> patch [s] | normal [s] | command
>>>>>>>>> -----------------------------------------------------------------------
>>>>>>>>>
>>>>>>>>> 1.468 | 2.287 | ipa user-add jsmith --firt=john
>>>>>>>>> --last=smith
>>>>>>>>> 1.658 | 2.235 | ipa user-del jsmith
>>>>>>>>> 1.624 | 2.204 | ipa dnsrecord-find example.com
>>>>>>>>>
>>>>>>>> Thanks for submitting the patch. Ondra, just please provide the
>>>>>>>> patch in
>>>>>>>> proper format (exported via command `git format-patch -M -C
>>>>>>>> --patience
>>>>>>>> --full-index -1' which I sent you earlier).
>>>>>>>>
>>>>>>>> Martin
>>>>>>>>
>>>>>>>>
>>>>>>> Sorry, correct version attached
>>>>>>
>>>>>> Wow, this is very impressive, great job Ondra and Honza!
>>>>>>
>>>>>> Martin, ACK from me but I'd like a second opinion. The patch is very
>>>>>> straightforward and clean, just want to make sure we're not missing a
>>>>>> corner case.
>>>>>>
>>>>>> I ran the self-tests and didn't see any problem there.
>>>>>>
>>>>>> Before pushing please add the ticket # to the commit.
>>>>>>
>>>>>> rob
>>>>>>
>>>>>
>>>>> I've just remembered that special methods aren't looked up through
>>>>> __getattribute__ (see the note at
>>>>> http://docs.python.org/reference/datamodel.html#more-attribute-access-for-new-style-classes).
>>>>>
>>>>> That might possibly cause problems in Command and subclasses,
>>>>> because it
>>>>> uses __call__.
>>>>>
>>>>> Honza
>>>>>
>>>>
>>>> This should be investigated. All our unit tests are OK though.
>>>>
>>>> I am also thinking if we shouldn't do this optimization on CLI side
>>>> only
>>>> as the original ticket suggests. As server side loads and finalizes all
>>>> plugins just once at httpd start, performance should be OK. Plus, we
>>>> would know right after the start that we are ready and there is no
>>>> problem with finalizing any component.
>>>>
>>>> Martin
>>>>
>>>
>>> As we discussed earlier - are you sure it works that way?
>>
>> Yes.
>>
>>> I did a few
>>> Django-based applications before and I'm pretty sure all mod_wsgi does
>>> is caching the bytecode, which is run from the beginning for each
>>> request, like what CLI does.
>>
>> I did few Django application myself too. Its all about how mod_wsgi is
>> configured. You may want to learn more about WSGIDaemonProcess and
>> WSGIImportScript directives.
>>
>> In IPA, we have API already prepared and finalized for every incoming
>> request.
>>
>>>
>>> If it does work the way you described, the worst thing that could happen
>>> is that the plugins would be finalized the first time they are used and
>>> then stay initialized until the server terminates.
>>>
>>> Honza
>>>
>>
>> I know, I was just being conservative. As this optimization has no
>> effect on server performance I considered it a little bit safer to load
>> everything at start and be sure we are OK. But its not that hard
>> requirement.
>>
>> Martin
>>
>
> OK, thanks, just wanted to be sure.
>
> Honza
>

Added finalization for __call__ and the check for CLI. Patch attached.

Honza

-- 
Jan Cholasta
-------------- next part --------------
A non-text attachment was scrubbed...
Name: freeipa-jcholast-57-lazy-finalize.patch
Type: text/x-patch
Size: 2674 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/freeipa-devel/attachments/20111031/2db3cbcb/attachment.bin>