[Pki-devel] [PATCH] 4 Replaced byte-to-char & char-to-byte converters.

[Endi Sukma Dewata]

On 1/5/2012 3:00 PM, Endi Sukma Dewata wrote:
> About the performance, I compared the results of #9 before and after
> conversion to Charset, there are some differences but I don't think the
> difference is significant.

To clarify, in patch #9 I wrote simple tests for measuring the encoding 
and decoding times for each converter (identified by the tag) to process 
a set of chars/bytes 10k times.

   for (int i=0; i<10000; i++) encode(tag, chars);

   for (int i=0; i<10000; i++) decode(tag, bytes);

Each test is done twice, one using Mozilla JSS library and another using 
our internal library which is based on sun.io before the upgrade and 
java.nio (Charset) after the upgrade.

Below is the result. The 2 numbers in each category correspond to the 
execution times before and after the upgrade.

BMP String Encoding time:
  - JSS     : 210 ms   212 ms
  - Internal: 281 ms   319 ms
BMP String Decoding time:
  - JSS     : 186 ms   196 ms
  - Internal: 212 ms   149 ms
IA5 String Encoding time:
  - JSS     : 42 ms    101 ms
  - Internal: 46 ms    110 ms
IA5 String Decoding time:
  - JSS     : 93 ms    118 ms
  - Internal: 84 ms    37 ms
Printable String Encoding time:
  - JSS     : 26 ms    113 ms
  - Internal: 48 ms    86 ms
Printable String Decoding time:
  - JSS     : 37 ms    49 ms
  - Internal: 40 ms    38 ms
Teletex String Encoding time:
  - JSS     : 18 ms    36 ms
  - Internal: 43 ms    103 ms
Teletex String Decoding time:
  - JSS     : 16 ms    16 ms
  - Internal: 23 ms    32 ms
Universal String Encoding time:
  - JSS     : 74 ms    71 ms
  - Internal: 30 ms    49 ms
Universal String Decoding time:
  - JSS     : 80 ms    76 ms
  - Internal: 42 ms    18 ms
UTF-8 String Encoding time:
  - JSS     : 13 ms    31 ms
  - Internal: 28 ms    59 ms
UTF-8 String Decoding time:
  - JSS     : 37 ms    51 ms
  - Internal: 46 ms    52 ms

The JSS tests are supposed to produce similar numbers before and after 
the upgrade since they are executing the same code, but that's not 
always the case, see IA5 encoding and printable encoding times. This is 
probably due to the changing nature of machine load at the time of the 
execution.

If we look at the internal numbers, some go up and some go down. We 
cannot really say whether it's an improvement or degradation since the 
JSS numbers that we use as a baseline aren't always stable. Also since 
different converter uses different code, there isn't a single number 
that can describe the overall behavior.

So all I can conclude is that the upgrade to the internal library does 
not show any significant performance degradation that we should be 
concerned about at this point. If we want to get real statistical 
numbers (e.g. % improvement over the old library or JSS) we will have to 
define the test scope, generate better test data, and execute the tests 
systematically with varying loads. I'm not sure if we want to do that at 
this stage.

-- 
Endi S. Dewata