[lvm-devel] lvm and locales memory issue

[Zdenek Kabelac]

On 23.2.2010 17:28, Jakub Jelinek wrote:
>> Another note could be - Ubuntu does not even use locale-archive file and uses
>> locales on per file basis - so now I'm getting curious, where are the tests,
> 
> Not everything Ubuntu does is necessarily a good idea.
> 
>> that proves that Fedora gets some measurable performance advantage?
> 
> Try something trivial, like:
> #include <locale.h>
> 
> int
> main (void)
> {
>   int i;
>   for (i = 0; i < 1000000; i++)
>     switch (i % 5)
>       {
>       case 0: setlocale (LC_ALL, "C"); break;
>       case 1: setlocale (LC_ALL, "en_US.UTF-8"); break;
>       case 2: setlocale (LC_ALL, "cs_CZ.UTF-8"); break;
>       case 3: setlocale (LC_ALL, "fr_FR.UTF-8"); break;
>       case 4: setlocale (LC_ALL, "de_DE.UTF-8"); break;
>       }
>   return 0;
> }
> 
> With locale-archive 1.362s, without, using locale files, 10.355s.  And


>From looking into the code and keeping in mind we need to handle these
millions of switches per second as a killer feature - there seems to be nice
way -  instead of doing one large mmap call - how about doing several
smaller sized mmap just for regions needed by given locale.

Once mmap-ed - it will stay in program's memory till its exit just like now...

So - instead of one large 100MB mmap - we would end for this 'benchmark'
with maybe 8-10 mmap calls per setlocale - so let's say 50 mmap calls for
small memory regions. (or less - depends on how the locale-archive is
organized - maybe everything except LC_CTYPE & LC_COLLATE could be in on
mmaped area)

As a bonus - for the most common use-case - it might eat less pgt entries
(I assume its ~190kb for x86_64 and 100MB file)
On the opposite site - user of all locales at once would waste memory by
having some pages mmmaped multiple times....

Zdenek