[Pki-devel] [PATCH] 4 Replaced byte-to-char & char-to-byte converters.

[Nathan Kinder]

On 01/06/2012 04:11 PM, Endi Sukma Dewata wrote:
> On 1/5/2012 3:00 PM, Endi Sukma Dewata wrote:
>> About the performance, I compared the results of #9 before and after
>> conversion to Charset, there are some differences but I don't think the
>> difference is significant.
>
> To clarify, in patch #9 I wrote simple tests for measuring the 
> encoding and decoding times for each converter (identified by the tag) 
> to process a set of chars/bytes 10k times.
>
>   for (int i=0; i<10000; i++) encode(tag, chars);
>
>   for (int i=0; i<10000; i++) decode(tag, bytes);
>
> Each test is done twice, one using Mozilla JSS library and another 
> using our internal library which is based on sun.io before the upgrade 
> and java.nio (Charset) after the upgrade.
>
> Below is the result. The 2 numbers in each category correspond to the 
> execution times before and after the upgrade.
>
> BMP String Encoding time:
>  - JSS     : 210 ms   212 ms
>  - Internal: 281 ms   319 ms
> BMP String Decoding time:
>  - JSS     : 186 ms   196 ms
>  - Internal: 212 ms   149 ms
> IA5 String Encoding time:
>  - JSS     : 42 ms    101 ms
>  - Internal: 46 ms    110 ms
> IA5 String Decoding time:
>  - JSS     : 93 ms    118 ms
>  - Internal: 84 ms    37 ms
> Printable String Encoding time:
>  - JSS     : 26 ms    113 ms
>  - Internal: 48 ms    86 ms
> Printable String Decoding time:
>  - JSS     : 37 ms    49 ms
>  - Internal: 40 ms    38 ms
> Teletex String Encoding time:
>  - JSS     : 18 ms    36 ms
>  - Internal: 43 ms    103 ms
> Teletex String Decoding time:
>  - JSS     : 16 ms    16 ms
>  - Internal: 23 ms    32 ms
> Universal String Encoding time:
>  - JSS     : 74 ms    71 ms
>  - Internal: 30 ms    49 ms
> Universal String Decoding time:
>  - JSS     : 80 ms    76 ms
>  - Internal: 42 ms    18 ms
> UTF-8 String Encoding time:
>  - JSS     : 13 ms    31 ms
>  - Internal: 28 ms    59 ms
> UTF-8 String Decoding time:
>  - JSS     : 37 ms    51 ms
>  - Internal: 46 ms    52 ms
>
> The JSS tests are supposed to produce similar numbers before and after 
> the upgrade since they are executing the same code, but that's not 
> always the case, see IA5 encoding and printable encoding times. This 
> is probably due to the changing nature of machine load at the time of 
> the execution.
>
> If we look at the internal numbers, some go up and some go down. We 
> cannot really say whether it's an improvement or degradation since the 
> JSS numbers that we use as a baseline aren't always stable. Also since 
> different converter uses different code, there isn't a single number 
> that can describe the overall behavior.
>
> So all I can conclude is that the upgrade to the internal library does 
> not show any significant performance degradation that we should be 
> concerned about at this point. If we want to get real statistical 
> numbers (e.g. % improvement over the old library or JSS) we will have 
> to define the test scope, generate better test data, and execute the 
> tests systematically with varying loads. I'm not sure if we want to do 
> that at this stage.
I agree that the numbers don't show anything that we should be concerned 
with at this point.  If your tests from patch #8 prove encoding/decoding 
correctness, I think we should move forward.  We should file a trac 
ticket for general performance profiling and target it for a future 
milestone (likely a few months from now).

-NGK