[Freeipa-devel] [PATCH] test speedup patch

[Pavel Zuna]

On 01/19/2011 04:17 AM, Rob Crittenden wrote:
> Rob Crittenden wrote:
>> Rob Crittenden wrote:
>>> Attached is a rough cut of a patch to try to speed up the cli a little
>>> bit. Basically in production mode it will skip some things during
>>> initialization.
>>>
>>> My concept is that we develop in mode != production and release in mode
>>> == production.
>>>
>>> I managed to knock a second or so off time to do a user-show on average.
>>>
>>> There may be some other things we can do to speed things up, I'm still
>>> looking. Some feedback on the approach would be appreciated.
>>>
>>> Note that I've completely ruled out SSL/Negotiate. I did my testing on
>>> lite-server which doesn't use SSL or Negotiate and it was STILL taking
>>> on average 3-4+ seconds per command. The server side was consistently
>>> taking < 1 second to complete.
>>>
>>> rob
>>
>> oh, and the patch.
>
> I ran a couple of moderate tests this evening that executed 42 separate
> operations like add, delete, and managing group membership. I ran this
> 10 times each on 2 identical VMs, one with a bit older code and one with
> this patch then averaged the times.
>
> With the patch the average was 1.3 seconds per operation, without 2.6. A
> 50% improvement is more than I expected, I saw a 33% improvement on
> individual runs. I'll keep at it but this seems promising. I was also a
> bit surprised that the average time without the patch was so low, I was
> expecting something over 3 seconds.
>
> Specifically what this patch does is it avoids doing some
> self-validation. There is some amount of risk that the framework could
> blow up but in a deployed situation I think the risk is rather low.
>
> A side-effect of the API tester makeapi is that it loads the framework.
> We can force it to be run in production mode so the product shouldn't be
> buildable if it has inconsistencies.
>
> rob
>

I find it hard to believe this patch causes such a big improvement in 
performance. Especially the parts skipping asserts, that shouldn't be 
significantly slower than your average ifs. Instance locking shouldn't be a time 
consuming operation either.

Bypassing check routines for parameter namespaces might provide a performance 
boost as it is called for every single plugin we have (~250). On the other hand, 
it is only used for positional arguments and most plugins only have 1 or 2 of those.

Personally, I would do some more tests on a single machine, because there's no 
guarantee, that two VMs with an identical image have the same performance.

If it really provides a significant improvement, than it's awesome, because I 
like the philosophy of this patch. It removes self-checking and instance 
locking, that is completely useless in a production environment and kind of 
limiting in non-production. I think there's more places likes this in the framework.

Long story short:
It's improbable, but not impossible, for the changes introduced by this patch to 
cause such a big performance improvement. Even if it doesn't, the patch is still 
good.

Pavel