Asian Squirrelmail trouble

[Warren Togami]

Hyde Yamakawa wrote:
> Hello, sir,
> 
> I am the one asked to fix the Japanese encoding problem of
> squirrelmail-1.4.6-3.fc4 and thank you again for your fixing.
> 
> I have two more concerns about this squirellmail.
> One is the file name in Japanese is mis decoded when save the
> attached file.
> And another is mis decoding part of Japanese in body sometimes.
> 
> Should I report these to bugzilla also?
> 
> I am just wondering if these are known and inpossible to fix
> because they have been existing quite long time.
> 
> I am sorry but if you can speak Japanese so I wrote in English.
> Which is better English or Japanese?
> I am Japanese.
> 
> Regards,
> Hyde
> 

Sorry, my Japanese language skill is very poor so I must communicate in 
English for now.

We are not happy about squirrelmail because it is poorly maintained by 
the upstream project and "stuck in the past" with very little progress 
and very poor support for non-Western language encodings.  I *want* to 
replace squirrelmail with something more modern, with strong commercial 
and community support behind the project, and well designed 
international encoding support.  But I don't have time to seek, test, 
and package alternatives to squirrelmail currently.

One of our European engineers dwmw2 tried to improve the Unicode 
situation in our squirrelmail with the current scripted conversion hack 
in an attempt to workaround upstream squirrelmail's horrible 
localization policy where their encodings are inconsistently mixed. 
This improved things a bit only for some European languages, but not the 
Asian languages.

As you may be well aware, it is currently infeasible to expect the Asian 
countries to use only Unicode encoding due to the constantly moving 
standards and different glyphs in different languages using the same 
code-points, among other problems.

In order to support the current reality of language specific encodings 
in Asian languages, it would require native developers to "fix" the 
problem and submit patches upstream.  This however is going to be a 
difficult problem because there is no easy agreement among developers 
and users in those countries about what exactly is the "best" way of 
doing it.  To make matters worse, if you look at the current 
squirrelmail code it is full of ugly hacks in support of Asian encodings 
that bafflingly make little sense.  For example, why the heck is the web 
interface in EUCJP encoding while e-mail sent and received is 
ISO-2022-JP by default?

Within Japan for example, currently users are using mainly ISO-2022-JP, 
which is the default encoding in Microsoft Windows XP and below.  I 
heard that Windows Vista will be UTF-8 by default with support for the 
new government mandated JIS X 0213:2004 standard in the private area. 
Fedora has been UTF-8 for a long time now.

I suspect this is a cause of "wrong" encodings when saving files from 
squirrelmail because some users want the Windows XP default of 
ISO-2022-JP encodings and other users want UTF-8 encodings?  I don't 
know for certain and don't have time to research this myself.

In any case, these encoding troubles for Asian software are difficult to 
solve due to the bewildering reality of conflicting requirements and 
necessity of supporting both legacy and new systems.  Squirrelmail will 
probably never be fixed by the current upstream project.  If you want it 
to be fixed, then native Japanese developers will need to work on it, 
then convince the upstream project to accept their patches.

Sorry I cannot be more helpful.

Warren Togami
wtogami at redhat.com