RFC: i18n proposal

[Jeff Johnson]

On Tue, Jul 29, 2003 at 11:39:47PM +0200, Göran Uddeborg wrote:
> Jeff Johnson writes:
> > The only difference from the current specspo implementation (which uses
> > dgettext) is:
> > 	you are moving the package derived retrieval key from the
> > 	2nd MSGID arg to the 1st DOMAINNAME arg without specifiying
> > 	what is to replace MSGID.
> 
> The specspo implementation creates one huge database of descriptions
> for all and only packages of one release.  My intention was to try to
> envision a scheme, inspired by the ideas in Havoc's letters, where
> this was split up per package instead.
> 

(under seperate cover)

> > If you are hoping to use the MSGID from package headers as the MSGID key,
> > then your solution is inferior to the current specspo implementation
> > because the Group/Summary/Description then *must* be built into packages.
> 
> That was how I thought about it.  But I guess it is not really
> critical.  You could continue the current practice of using a po-file
> for a key-value lookup, and still have one po file per package as I
> suggested.
> 
> I haven't fully understood why it is such a good idea.  I thougt
> gettext was designed to avoid having opaque keys, and instead use the
> English language message directly as the key.  And this seems to go
> contrary to that, back to the catgets() inteface.
> 

The basic reason for splitting msgid from msgstr is that the docs team
writes the msgid's, the translation team supplies msgstr's; hence
package i18n is a little different than traditional xgettext on *.c.

> 
> > 	a) msgid's, not just msgstr's, need to be handled. msgid's are not
> > 	necessarily constant,
> 
> If I update specspo, it may result in that descriptions of already
> installed packages change or disappear.  Is it this sense of "not
> ... constant" you refer to?
> 

Unlike, say, error messages with xgettext *.c, summary/description/group
are much softer, so text divergence is acceptable.

Yes, that's a feature, at least for the docs team editors, because text
can be updated in the field without having to wait for "make world"
\of the next distro release.

It's also a feature for translation teams, which are no longer subject
to the tyrrany of a distro release deadline, specspo could/should be released
as needed (i.e. when msgid's change from docs team), not when there is
a distro release.

Sure, specspo needs to be burned on the CD, there are alternative
update pathways that could/should be used for specspo.

Again, these are process, not implementation, solutions so my feature
may very well be your bug ;-)

> I consider that a bug.  One of the bugs I'm trying to solve here.
> 

> If I do "rpm -qip xmms-1.2.7-1.i386.rpm", with specspo installed, I
> get a description not mentioning MP3.  If I uninstall specspo, or
> change to locale C or so, I get the package's original description,
> mentioning MP3.
> 
> Similarily, if I to "rpm -qi mpg321" on a system which still has this
> package, but current specspo, I don't get the description translatied.
> There was a translation, but it is not part of current specspo.
> 
> In both cases, the messages, and translations of them, which existed
> at the time of package generation is better than the current
> situation.  They ought to come and go with the package itself.
> 

Confusing? You betcha, many developers are suprised when they look
for their edits to spec files to appear with "rpm -qi foo".

But that's a process (i.e. send patch to docs team, not edit spec file)
rather than a packaging problem.

rpm can go several ways here.

If I go the way I want, then description/summary/group will be removed
from packages entirely and problem will cease to exist.

Controversial? You betcha, as it appears that I'm trying to deny users
access to descriptions of packages when in fact something else is intended.

> >       nor is there (or at least gas there been)
> > 	sufficient time late in a release cycle to manage "make world"
> > 	rebuilds to include i18n, too many things can break.
> 
> That could of course be a problem.
> 
> To me it seems it would be possible to freeze descriptions of the new
> packages earlier.  Like already by now in the current cycle.  And also
> make them available for translation at this time.  Then they could be
> there when the final build is done.
> 

We've tried freeze, there's often too much package rearrangement during
a release cycle for an early freeze to be effective.

Might work now, but there's still lots of package churn.

> > 	c) the goal must be to have i18n tags in metadata, fetching from
> > 	the payload is a costly decompression.
> 
> I must have misunderstood something.  In message
> <20030724114226.Q18401 at devserv.devel.redhat.com> you said:
> 
> > IMHO: i18n does not belong in rpm metadata anymore than i18n belongs in
> > tar/cpio headers. Keep i18n out of packages, please.
> 
> I understood that to mean that you did not want the translations in
> the metadata.  So I tried to sketch things obeying that, putting them
> elsewhere.
> 

Sorry, my bad. There are 2 issues here:

	a) using already existing metadata tag values as retrieval keys
	for i18n from somewhere.

	b) the somewhere should not be the package payload because that
	will slow down the installer while the payload is read, looking
	for description/summary/group. In fact, because of genhdlist,
	anaconda does most of it's work without payloads being available.
	
> But maybe I misunderstood this comment.  If so, I guess one could try
> to extend the current "%description -l ...".  As I understand it, this
> does become package metadata; at least it winds up in the Packages
> database file.
> 
> As you pointed out, this lacks coding information.  Also, putting all
> translations in the spec file is not very convenient.  They need to be
> fetched externally at build time.  But if this is an acceptable
> method, these problems probably could be solved.
> 

And, yes, the old means of adding text to spec files still exists,
will probably exist forever. Red Hat has not used this since 6.1,
and is very unlikely to return to this mechanism. OTOH, it does mostly
solve the 3rd party single package i18n problem, the only thing that
breaks is when there are package name collisions, specspo is preferred to
header content. I question whether identically named package is really
3rd party packaging, but making the header tag retrieval order configurable
(i.e. prefer header over domains, or vice versa) is one approach if that
too needs solving.

> > Sure your scheme makes sense, but is possibly different than what rpm
> > has been doing since 6.2. There are consequences for having to support
> > multiple schemes for the indefinite future, not the least of which is
> > the complexity involved.
> 
> Point taken.
> 
> > IMHO the apparent driving force for revisiting an already solved problem
> > is the perceived need to accomodate 3rd party package group/summary/description.
> > 
> > Personally, I have yet to meet (or hear of) any single person who has even
> > attempted the translation.
> 
> Are we talking about translations of descriptions at all?  Or only
> through external po files?

I was referring to summary/description/group.

> 
> I always include Swedish versions when I build packages of my own.
> (After rhcontrib.bero.org died, I've mostly circulated them among
> friends.  Some could probably still be found out there; rpmfind found
> a stickers package for example.)
> 
> I may be uncommon doing this.  But then again, including translations
> in the SPEC file is not very convenient.  And is "%description -l"
> documented anywhere except for the change log and the source code?
> There is a chance not everybody is aware of the possibility.  :-)
> 

Again, I'm not at all averse to improving the subtleties of i18n handling
in packages as long as rpm package format doesn't have to change; afaict
the existing specspo mechanism can be used to have per-package domains
quite easily.

73 de Jeff

-- 
Jeff Johnson	ARS N3NPQ
jbj at redhat.com (jbj at jbj.org)
Chapel Hill, NC