anonym:
Mark Smith wrote:
On 3/10/14, 1:35 PM, Lunar wrote:
The french build is broken. I have an error message “undefined XML entity” on chrome://torlauncher/content/network-settings-wizard.xul line 130, column 43 on &torSettings.bridgeHelp;
There are some new strings in Tor Launcher that need to be translated. I don't know exactly what the interim process is since Mike has always taken care of localization issues for us, but I think English strings are usually pulled in for the untranslated entities.
As far as I can tell, DTD translation files are naively "applied" to the XUL files, without falling back to English for missing entities. If an entity is missing in the DTD, the resulting XUL keeps the entity name as-is, producing invalid XUL like "<tag>&entity_name</tag>". This explains the fatal error that lunar reported.
Problem 1: The above means that when a *new* string is introduced, all existing translations are broken until they are re-imported via `localization/import-translations.sh`, which depends on Transifex having added the corresponding entity to all locales' DTD files. It looks like the latter requires pushing the commits with the new string, and then wait a bit. If that's correct, it makes this situation quite awkward for anyone lacking push rights to your upstream repo, as they cannot package a Tor Launcher with working non-English locales without manually updating every single DTD file.
Yes, we have to wait for transifex to at least update the placeholders for us.
To make problem 1 less of a problem we could use a tool which can sync all non-English DTD:s to the English one. It be invoked via something like `make update-translations`, making it possible refresh the DTD files so one can package a Tor Launcher that works for all locales without having to involve Transifex (that can wait until right before release time). I'm quite ignorant of XML/DTD stuff, but clearly Transifex must make use of a tool like that. Any ideas about this?
Yes, however, it takes a few hours, and has had some issues (which they are in the process of resolving. See https://trac.torproject.org/projects/tor/ticket/9144).
I am not sure if their DTD merge tool is open source/available. Probably?
Problem 2: Another issue is that `chrome.manifest` doesn't list English as the first locale. If it did, English would be the fallback for locales not listed in `chrome.manifest` (note that this would not solve problem 1!). For the record, the current fallback is `af`, which doesn't even have a directory in `chrome/locale`, so all unlisted locales are broken.
Ah, I was actually not aware that the first locale was the fallback. Is it per-entity, or is it only used as a fallback if a locale is entirely absent? It sounds like only if a locale is absent?
Problem 3: However, `chrome.manifest` has a pretty exhaustive list of locales, so the broken fallback isn't used in many cases. Unfortunately most of them are pointing either to non-existing directories in `chrome/locale` (fatal error similar to the one from problem 1), and to existing directories which contain translations that are pretty much guaranteed to be outdated (and hence probably hit by problem 1) because they're not updated by `localization/import-translations.sh`. Hence all these locales will experience similar errors to what lunar reported.
For problems 2 and 3 I think some `make` automation can improve the situation quite a bit. See the attached patch for something like that.
Right. This sounds like a good start. I would prefer only one list of locales to update though.. Ideally in a form that makes it easy to cut+paste from the TBB list (ie an env var updated in two places like we have now).
In either case, we should only list locales supported in transifex though, it sounds like.
I suppose you should see all this as an inspiration for a real fix, as I'm not even sure how you want the translation process to work in the end (perhaps you're fine with the current situation?). For the same reason I've refrained from opening any bugs at this point.
For the record, the current plan for Tails 0.23 (due on 2014-03-19) is to ship a Tor Launcher packaged with my patch applied, since the current state of things breaks Tor Launcher in many locales we care about in Tails:
https://labs.riseup.net/code/issues/6885(Well, actually Tor Launcher is broken in *all* non-English in Tails due to problem 1 alone, since I introduced a new string. However, your commit a634ce6 fixed this, and my patch fixes the remaining issues, but this still shows the severity of problem 1 for downstreams wishing to package Tor Launcher with potential modifications, like Tails.)
I am very interested in arriving at a solution that works for both of us out of the box, but I also want to maintain the ability to make it straightforward to verify that the list of locales in TBB, Torbutton, and Tor Launcher are the same. This is most easily done if we have a single line of locales in an env var.
Otherwise, your patch looks like a good direction to head in.