[Scummvm-devel] Improving translation support in ScummVM

[Johannes Schickel]

On 06/24/2010 12:25 AM, Thierry Crozat wrote:
> Hi all,
>
> The introduction of translations is a great step forward, but with it some issues have crept up. I am looking for ideas or suggestions on how to solve some of these.
>
> The first one is the language selection in the GUI. The current list of language locale (e.g. ru_RU, de_DE, fr_FR) is not very user friendly. I can work on changing this, but before that we need to decide what we want to see in the GUI.
> * Can we drop the country and just list language names (e.g. "Russian", "German", "French") or do we want to accept translations in the same language but different countries (e.g. "French (France)" and "French (Canada)")? I would be in favor of displaying only the language.
>    

We should at least definitely keep the language codes as internal data, 
else it's hard to auto setup the language based on the setup locale for 
example. What could be done on the other hand is to extract the language 
string from the .po file in tools/po2c and store it in the same 
structure as we also do currently with the charset. That will require 
some perl knowledge (and I also propose to update to the development 
po2c script, which also uses a code formatting more similar to ours), 
but I guess that's not too hard to achieve.

Along with it all the tables in the po2c script's output could be made 
const too....

> * In which language should we display the language string? Always in english (e.g. "English", "French", "German") or in the native language for that string (e.g. "English", "Français", "Deutsch")? Maybe we would have an issue with the later as it means we might need characters that are not supported by the encoding of the currently set languages (e.g. the 'ç' character for french or the cyrillic characters for Russian). Maybe we can suppose everybody will know english just well enough to find the language he wants if the list is in english.

I guess English would be best due to our font limitations, of course for 
external translations we can't assume what they specify in case we do as 
I proposed above :-).

> Second issue: the layout is too tight and static and sometime we have to jump through hoops to find a translation that fits in the current layout (and this can result in an approximative translation). For example the current width for option labels (110 pixels in the modern theme if I understood it correctly) make it difficult to find a correct translation for some of the option labels. I can think of several ways to fix that:
> * slightly modify the modern theme so that it gives a bit more place to option labels. The current theme was obviously tweaked to work well with english, but does not work so well with other languages. But this look like hiding the dirt under the carpet. In the end the problem will still be there.
>    

Actually giving the options button more space than the others would kill 
the layout in general, i.e. that all buttons have the same size. So I'm 
not sure whether that's even a good temporary solution....

> * make a more dynamic layout where the size of labels is not hardcoded but defined at runtime depending on the text to display (similar to the Qt QLayout mechanism). Is that possible with the current layout system in ScummVM? But then we would also probably need a grid layout to make the options properly aligned without fixing the width of the option labels.
>    

I don't think our layouting engine takes the text into consideration.... 
but on the other hand I'm not entirely sure currently either. Vincent 
any idea here? :-P

> * support language dependent versions of the same theme. We could for example provide a version of the scummmodern themes (or other themes) tweaked for the language of a translation. For example if there is a scummmodern.zip and scummmodern_fr_FR.zip files in the theme path, it would only list ScummVM Modern theme once but used one or the user depending on the selected languages (in that case the scummmodern_fr_FR.zip one for french and scummmodern.zip for all other languages).
>    

Probably the easiest way out....

> Other suggestions are welcome.
>
> Also have you noticed other outstanding issues with the translation support (other than non latin1 font support)?
>    

I noticed that when compiled on Win64 with MSVC(9) targeting Win64, I 
got a ScummVM lock up when using the hu_HU translation when entering the 
Options menu. It all ended up in ScummVM taking up more and more memory, 
but I recently saw some fix for Common::String's memory management, 
maybe that was related. I did not try that since then.

I am not sure what you understand by "issues" I actually see some other 
issues with the translation support, which are not related due to 
missing data like with the fonts:

1) Some of the locale strings use %d etc. to add numbers, strings etc. 
to the output. While that might be fine, the major problem here is: the 
order is fixed, so it might be hard for translators to take grammar or 
the like into account when doing translations for those messages.

2) We definitely lost all the ability of format type checking (%d), due 
to the use of the "_(x)" etc. macros. This might result in missing 
format arguments in our code and of course in case we use the 
translation strings directly as format arguments (which we obviously 
need when we have %d etc. in them!) we might allow for bad formatting 
arguments too. For example this way we might allow stack overwrites with 
setup translation messages, by including %n for example.

Related here: In case someone wants to use "%" in his strings he *might* 
require to use %% in case we pass it directly as format argument OR just 
use "%", in case we use it this way: sprintf(output, "%s", 
_("translation here"));

3) The whole charset idea. Personally I'm not a fan of letting every 
translation use its own charset, which I stated on one of the patch 
tracker items back in the days too, but either nobody cared or nobody 
noticed that (and/or I forgot about the replies I got ;-)... Anyway my 
major problem with that is:

Now we have an even greater mess with charsets:

First of all our OSystem API's setWindowCaption is fixed to LATIN-1, 
i.e. game titles will still not be translatable properly, so the user 
might even be confused why his window title looks off, even though he 
setup a correctly looking name in ScummVM.

Also our input charset is not defined at all AFAIK, thus we can't even 
assume the user can properly enter save names with our GUI in his 
language. Which is also a bad thing for people who want "proper" 
language support.

Of course like you already noticed up there, we can not specify the 
language in its own language due to missing characters etc. While we 
might just want to assume everyone knows enough English, it's still not 
a nice solution. At least just take Wesnoth (http://www.wesnoth.org) it 
uses the native language name in its selection dialog too and I think 
most (all?) of the applications I used in my life did feature that too. 
So for a more "proper" translation support I would suggest to 
investigate the option of using some uniform charset again.

4) I think we still only support left-to-right languages, while of 
course that might be fine, it might feel like kind of a discrimination 
to people with a mother language, which is written right-to-left.

// Johannes