From alikins at redhat.com Mon May 21 16:33:14 2012 From: alikins at redhat.com (Adrian Likins) Date: Mon, 21 May 2012 12:33:14 -0400 Subject: [zanata-users] zh_Hans_CN zh_CN, zanata.xml, etc Message-ID: <4FBA6E4A.6080902@redhat.com> Trying to track down some weirdness we are seeing with the translation process for subscription-manager (https://translate.zanata.org/zanata/project/view/subscription-manager) and in particular the 0.99.x branch. Long story short, we noticed the pt_BR translations we had were ancient, despite pulling from zanata for every release. Eventually we figure out the zanata.xml got when we set up the branch is no longer the current one. ie, we had one like: https://translate.zanata.org/zanata/ subscription-manager 0.99.X as bn-IN de-DE es-ES fr gu hi it ja kn ko ml mr or pa pt-BR ru ta-IN te zh-Hans-CN zh-Hant-TW The latest one from https://translate.zanata.org/zanata/project/view/subscription-manager/iter/0.99.X/config is different. Amongst other things, it get's weird of the weird pt-BR entry above, and now just includes a pt_BR (which is nice and 100%), and a pt_PT which is not as much (which is fine, since it's not a supported lang). So updating the zanata.xml, and doing a zanata pull fixes the pt_BR issues. Question 1: how often do the zanata.xml's change, and what's the best way to insure I have the latest one? But... doing that changes the list of po's included, included the zh_* langs. Question 2: Our supported languages include zh_CN and zh_TW, and this is how we have tested (LANG=zh_CN.UTF8, etc). The latest zanata.xml and zanata tool however creates zh_Hans_CN.po and zh_Hant_TW.po. Installing these breaks LANG=zh_CN.UTF8. LANG=zh_Hans_CN.UTF8 also breaks subscription-manager client code in a way that suggest it's not setting the text encoding correctly for that case. Should I be installing these different? What is the relation between zh_CN and zh_Hans_CN? (quick answer would be greatly appreciated, since we are close to freeze) Adrian From omaciel at redhat.com Mon May 21 17:52:56 2012 From: omaciel at redhat.com (Og Maciel) Date: Mon, 21 May 2012 13:52:56 -0400 (EDT) Subject: [zanata-users] [Chainsaw] zh_Hans_CN zh_CN, zanata.xml, etc In-Reply-To: <4FBA6E4A.6080902@redhat.com> Message-ID: <34f3fd9a-a9d6-43b9-b1a2-a552ff079ae8@zmail11.collab.prod.int.phx2.redhat.com> Having had the chance to see both pt_BR translations (as I don't have anything good to say about their (lack of) "quality", I won't say anything) for subscription-manager and katello currently done via Zanata, I'd like to ask another question if possible: * why are we not using the same translation mechanism and process as the Fedora project (i.e. https://fedora.transifex.net/)? Katello is a community project but all the translation effort is currently done outside the community. I assume that "having translations done before a release" probably plays a role, but I'd love to see the community being able to do this. -- Og Maciel Senior QA Engineer Red Hat, Inc. +1.919.754.4782 irc: omaciel From sflaniga at redhat.com Tue May 22 00:07:54 2012 From: sflaniga at redhat.com (Sean Flanigan) Date: Tue, 22 May 2012 10:07:54 +1000 Subject: [zanata-users] zh_Hans_CN zh_CN, zanata.xml, etc In-Reply-To: <4FBA6E4A.6080902@redhat.com> References: <4FBA6E4A.6080902@redhat.com> Message-ID: <4FBAD8DA.5020305@redhat.com> On 05/22/2012 02:33 AM, Adrian Likins wrote: > > Trying to track down some weirdness we are seeing with the > translation process for subscription-manager > (https://translate.zanata.org/zanata/project/view/subscription-manager) > > and in particular the 0.99.x branch. > > Long story short, we noticed the pt_BR translations we had > were ancient, despite pulling from zanata for every > release. Eventually we figure out the zanata.xml > got when we set up the branch is no longer the current > one. ie, we had one like: > > > > https://translate.zanata.org/zanata/ > subscription-manager > 0.99.X > > as > bn-IN > de-DE > es-ES > fr > gu > hi > it > ja > kn > ko > ml > mr > or > pa > pt-BR > ru > ta-IN > te > zh-Hans-CN > zh-Hant-TW > > > > The latest one from > https://translate.zanata.org/zanata/project/view/subscription-manager/iter/0.99.X/config > is different. Amongst other things, it get's weird of the weird pt-BR > entry above, and now > just includes a pt_BR (which is nice and 100%), and a pt_PT which is not > as much (which > is fine, since it's not a supported lang). > > So updating the zanata.xml, and doing a zanata pull fixes the pt_BR issues. > > Question 1: how often do the zanata.xml's change, and what's the best > way to insure > I have the latest one? Zanata will generate zanata.xml whenever you click that button. You can ignore it if you like, it's just there to help with creating zanata.xml initially. But basically, the contents will change when the chosen locales on the server change. NB: The zanata.xml generated by Zanata never includes any map-from attributes, like pt-BR. I don't know where that came from, you'd have to look at your source control history. If someone changes the locales associated with the project, they should update zanata.xml (probably by hand) and commit it to source control if they want to push/pull those locales. > But... doing that changes the list of po's included, included the zh_* > langs. Included in what? > Question 2: Our supported languages include zh_CN and zh_TW, and this is how > we have tested (LANG=zh_CN.UTF8, etc). The latest zanata.xml and zanata tool > however creates zh_Hans_CN.po and zh_Hant_TW.po. Installing these > breaks LANG=zh_CN.UTF8. LANG=zh_Hans_CN.UTF8 also breaks > subscription-manager > client code in a way that suggest it's not setting the text encoding > correctly for > that case. You need to restore the map-from attributes for those locales, ie zh-Hans-CN zh-Hant-TW This will cause zh-Hans-CN to be pulled down to zh_CN.po instead of zh_Hans_CN.po. > > Should I be installing these different? What is the relation between > zh_CN and zh_Hans_CN? Obviously they really represent the same thing, but zh_Hans_CN includes information about the use of the Chinese simplified script. zh_Hans is often better though, because it lets you use Chinese simplified script anywhere (eg Malaysia, Singapore), without having to tell the OS that you live in China. Unfortunately, locales like zh_Hans and zh_Hans_CN aren't well supported by runtime i18n libraries. > > (quick answer would be greatly appreciated, since we are close to freeze) > > > Adrian > -- Sean Flanigan Senior Software Engineer Engineering - Internationalisation Red Hat -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 554 bytes Desc: OpenPGP digital signature URL: From dchen at redhat.com Tue May 22 01:16:33 2012 From: dchen at redhat.com (Ding Yi Chen) Date: Mon, 21 May 2012 21:16:33 -0400 (EDT) Subject: [zanata-users] zh_Hans_CN zh_CN, zanata.xml, etc In-Reply-To: <4FBAD8DA.5020305@redhat.com> Message-ID: <30c216bb-6e4f-4010-9033-3c4cc51f27be@zmail13.collab.prod.int.phx2.redhat.com> > > The latest one from > > https://translate.zanata.org/zanata/project/view/subscription-manager/iter/0.99.X/config > > is different. Amongst other things, it get's weird of the weird > > pt-BR > > entry above, and now > > just includes a pt_BR (which is nice and 100%), and a pt_PT which > > is not > > as much (which > > is fine, since it's not a supported lang). > > > > So updating the zanata.xml, and doing a zanata pull fixes the pt_BR > > issues. > > > > Question 1: how often do the zanata.xml's change, and what's the > > best > > way to insure > > I have the latest one? > > Zanata will generate zanata.xml whenever you click that button. You > can > ignore it if you like, it's just there to help with creating > zanata.xml > initially. But basically, the contents will change when the chosen > locales on the server change. NB: The zanata.xml generated by Zanata > never includes any map-from attributes, like map-from="pt">pt-BR. I don't know where that came from, > you'd > have to look at your source control history. > > If someone changes the locales associated with the project, they > should > update zanata.xml (probably by hand) and commit it to source control > if > they want to push/pull those locales. > > Question 2: Our supported languages include zh_CN and zh_TW, and > > this is how > > we have tested (LANG=zh_CN.UTF8, etc). The latest zanata.xml and > > zanata tool > > however creates zh_Hans_CN.po and zh_Hant_TW.po. Installing these > > breaks LANG=zh_CN.UTF8. LANG=zh_Hans_CN.UTF8 also breaks > > subscription-manager > > client code in a way that suggest it's not setting the text > > encoding > > correctly for > > that case. > I have written a small utility named: zanata-util. It has a script, zanata_zanata_xml_make, address these kinds of problems by firstly download zanata.xml from server to retrieve the server supported locales, then provide the nessary map-from attribute by look up the existing .po files or output of locale -a > > Should I be installing these different? What is the relation > > between > > zh_CN and zh_Hans_CN? zh_CN: Chinese used in China zh_Hans: Simplified Chinese zh_Hans_CN: Simplified Chinese used in China Usually these 3 are exchangeable, unless you have some use cases such as zh_Hans_SG: Simplified Chinese used in Singapore. Wikipedia actually adopts this style of locale. -- Ding-Yi Chen Software Engineer Internationalization Group DID: +61 7 3514 8239 Email: dchen at redhat.com Red Hat, Asia-Pacific Pty Ltd Level 1, 193 North Quay Brisbane 4000 Office: +61 7 3514 8100 Fax: +61 7 3514 8199 Website: www.redhat.com Red Hat, Inc. Facebook: Red Hat APAC | Red Hat Japan | Red Hat Korea | JBoss APAC Twitter: Red Hat APAC | Red Hat ANZ LinkedIn: Red Hat APAC | JBoss APAC From alikins at redhat.com Tue May 22 13:19:17 2012 From: alikins at redhat.com (Adrian Likins) Date: Tue, 22 May 2012 09:19:17 -0400 Subject: [zanata-users] zh_Hans_CN zh_CN, zanata.xml, etc In-Reply-To: <4FBAD8DA.5020305@redhat.com> References: <4FBA6E4A.6080902@redhat.com> <4FBAD8DA.5020305@redhat.com> Message-ID: <4FBB9255.5090603@redhat.com> On 05/21/2012 08:07 PM, Sean Flanigan wrote: > On 05/22/2012 02:33 AM, Adrian Likins wrote: >> >> Long story short, we noticed the pt_BR translations we had >> were ancient, despite pulling from zanata for every >> release. Eventually we figure out the zanata.xml >> got when we set up the branch is no longer the current >> one. ie, we had one like: >> >> Question 1: how often do the zanata.xml's change, and what's the best >> way to insure >> I have the latest one? > Zanata will generate zanata.xml whenever you click that button. You can > ignore it if you like, it's just there to help with creating zanata.xml > initially. But basically, the contents will change when the chosen > locales on the server change. NB: The zanata.xml generated by Zanata > never includes any map-from attributes, like map-from="pt">pt-BR. I don't know where that came from, you'd > have to look at your source control history. > Interesting. Nothing revealing in the git logs, looks like the previous mapped to changes landed at the same time as the change from internal zanata to external zanata. Could have been done by hand, hard to say. > If someone changes the locales associated with the project, they should > update zanata.xml (probably by hand) and commit it to source control if > they want to push/pull those locales. Cool, that's what I ended up doing, so good that's the preffered approach. >> But... doing that changes the list of po's included, included the zh_* >> langs. > Included in what? Included in the zanata.xml (or more specifically, it was added to the latest zanata.xml from the server at some point, and wasn't there in the version we had in source control) > > You need to restore the map-from attributes for those locales, ie > > zh-Hans-CN > zh-Hant-TW > > > This will cause zh-Hans-CN to be pulled down to zh_CN.po instead of > zh_Hans_CN.po. > Cool, that's what I thought but figured I'd check. >> Should I be installing these different? What is the relation between >> zh_CN and zh_Hans_CN? > Obviously they really represent the same thing, but zh_Hans_CN includes > information about the use of the Chinese simplified script. zh_Hans is > often better though, because it lets you use Chinese simplified script > anywhere (eg Malaysia, Singapore), without having to tell the OS that > you live in China. > > Unfortunately, locales like zh_Hans and zh_Hans_CN aren't well supported > by runtime i18n libraries. Seems like it. Fortunately, things seem happy installing zh_Hans_CN and zh_Hant_TW as zh_CN and zh_TW, respectively. One more question, is there a good doc on what the "map-from" attributes should be? We previously mapped: de_DE -> de es_ES -> es bn_IN -> in ta_IN -> ta pt_BR -> pt It seems like at least pt_BR -> pt is somewhat frowned upon. I've added the maps from the zh_ langs. Should I add others? The version I currently have up for review only maps in cases where the zanata provided versions don't map to the supported RHEL6 langs (ie, the zh_* cases). ie, I currently have: as bn-IN de-DE es-ES fr gu hi it ja kn ko ml mr or pa pt-BR ru ta-IN te zh-Hans-CN zh-Hant-TW Does this seem correct? Thanks! From alikins at redhat.com Tue May 22 13:23:36 2012 From: alikins at redhat.com (Adrian Likins) Date: Tue, 22 May 2012 09:23:36 -0400 Subject: [zanata-users] zh_Hans_CN zh_CN, zanata.xml, etc In-Reply-To: <30c216bb-6e4f-4010-9033-3c4cc51f27be@zmail13.collab.prod.int.phx2.redhat.com> References: <30c216bb-6e4f-4010-9033-3c4cc51f27be@zmail13.collab.prod.int.phx2.redhat.com> Message-ID: <4FBB9358.1090002@redhat.com> On 05/21/2012 09:16 PM, Ding Yi Chen wrote: >>> The latest one from >>> https://translate.zanata.org/zanata/project/view/subscription-manager/iter/0.99.X/config >>> is different. Amongst other things, it get's weird of the weird >>> pt-BR >>> entry above, and now >>> just includes a pt_BR (which is nice and 100%), and a pt_PT which >>> is not >>> as much (which >>> is fine, since it's not a supported lang). >>> >>> So updating the zanata.xml, and doing a zanata pull fixes the pt_BR >>> issues. >>> >>> Question 1: how often do the zanata.xml's change, and what's the >>> best >>> way to insure >>> I have the latest one? >> Zanata will generate zanata.xml whenever you click that button. You >> can >> ignore it if you like, it's just there to help with creating >> zanata.xml >> initially. But basically, the contents will change when the chosen >> locales on the server change. NB: The zanata.xml generated by Zanata >> never includes any map-from attributes, like> map-from="pt">pt-BR. I don't know where that came from, >> you'd >> have to look at your source control history. >> >> If someone changes the locales associated with the project, they >> should >> update zanata.xml (probably by hand) and commit it to source control >> if >> they want to push/pull those locales. > >>> Question 2: Our supported languages include zh_CN and zh_TW, and >>> this is how >>> we have tested (LANG=zh_CN.UTF8, etc). The latest zanata.xml and >>> zanata tool >>> however creates zh_Hans_CN.po and zh_Hant_TW.po. Installing these >>> breaks LANG=zh_CN.UTF8. LANG=zh_Hans_CN.UTF8 also breaks >>> subscription-manager >>> client code in a way that suggest it's not setting the text >>> encoding >>> correctly for >>> that case. > I have written a small utility named: zanata-util. > It has a script, zanata_zanata_xml_make, address these kinds of problems by > firstly download zanata.xml from server to retrieve the server supported locales, > then provide the nessary map-from attribute by look up the existing .po files > or output of locale -a > Interesting, I'll grab it from epel and give it a shot. That sounds like exactly what I was looking for. Adrian From sflaniga at redhat.com Tue May 22 23:44:25 2012 From: sflaniga at redhat.com (Sean Flanigan) Date: Wed, 23 May 2012 09:44:25 +1000 Subject: [zanata-users] zh_Hans_CN zh_CN, zanata.xml, etc In-Reply-To: <4FBB9255.5090603@redhat.com> References: <4FBA6E4A.6080902@redhat.com> <4FBAD8DA.5020305@redhat.com> <4FBB9255.5090603@redhat.com> Message-ID: <4FBC24D9.3020306@redhat.com> On 05/22/2012 11:19 PM, Adrian Likins wrote: > On 05/21/2012 08:07 PM, Sean Flanigan wrote: >> On 05/22/2012 02:33 AM, Adrian Likins wrote: >>> >>> Long story short, we noticed the pt_BR translations we had >>> were ancient, despite pulling from zanata for every >>> release. Eventually we figure out the zanata.xml >>> got when we set up the branch is no longer the current >>> one. ie, we had one like: >>> >>> Question 1: how often do the zanata.xml's change, and what's the best >>> way to insure >>> I have the latest one? >> Zanata will generate zanata.xml whenever you click that button. You can >> ignore it if you like, it's just there to help with creating zanata.xml >> initially. But basically, the contents will change when the chosen >> locales on the server change. NB: The zanata.xml generated by Zanata >> never includes any map-from attributes, like> map-from="pt">pt-BR. I don't know where that came from, you'd >> have to look at your source control history. >> > Interesting. Nothing revealing in the git logs, looks like > the previous mapped to changes landed at the same time > as the change from internal zanata to external zanata. > Could have been done by hand, hard to say. Must have been. >> If someone changes the locales associated with the project, they should >> update zanata.xml (probably by hand) and commit it to source control if >> they want to push/pull those locales. > > Cool, that's what I ended up doing, so good that's the preffered > approach. > >>> But... doing that changes the list of po's included, included the zh_* >>> langs. >> Included in what? > Included in the zanata.xml (or more specifically, it was > added to the latest zanata.xml from the server at some point, > and wasn't there in the version we had in source control) > >> >> You need to restore the map-from attributes for those locales, ie >> >> zh-Hans-CN >> zh-Hant-TW >> >> >> This will cause zh-Hans-CN to be pulled down to zh_CN.po instead of >> zh_Hans_CN.po. >> > Cool, that's what I thought but figured I'd check. >>> Should I be installing these different? What is the relation between >>> zh_CN and zh_Hans_CN? >> Obviously they really represent the same thing, but zh_Hans_CN includes >> information about the use of the Chinese simplified script. zh_Hans is >> often better though, because it lets you use Chinese simplified script >> anywhere (eg Malaysia, Singapore), without having to tell the OS that >> you live in China. >> >> Unfortunately, locales like zh_Hans and zh_Hans_CN aren't well supported >> by runtime i18n libraries. > > Seems like it. Fortunately, things seem happy installing zh_Hans_CN > and zh_Hant_TW as zh_CN and zh_TW, respectively. > > One more question, is there a good doc on what the "map-from" > attributes should be? > > We previously mapped: > de_DE -> de > es_ES -> es > bn_IN -> in > ta_IN -> ta > pt_BR -> pt > > It seems like at least pt_BR -> pt is somewhat frowned upon. At one stage we were trying to keep country-specific locales out of the Zanata server config. de-DE and de usually 'mean' the same thing. pt-BR is different though, because pt and pt-BR are quite different. We're trying to move away from locale mapping, but there's some things that need to change in the server first. In the meantime, I recommend keeping mapping to a minimum. > I've added the maps from the zh_ langs. Should I add others? No, that should be fine. > The version I currently have up for review only maps in cases > where the zanata provided versions don't map to the supported > RHEL6 langs (ie, the zh_* cases). Yes, that's exactly right. > ie, I currently have: > > > as > bn-IN > de-DE > es-ES > fr > gu > hi > it > ja > kn > ko > ml > mr > or > pa > pt-BR > ru > ta-IN > te > zh-Hans-CN > zh-Hant-TW > > > Does this seem correct? Yes, that looks okay. -- Sean Flanigan Senior Software Engineer Engineering - Internationalisation Red Hat -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 554 bytes Desc: OpenPGP digital signature URL: