Project

General

Profile

To do #556

Refactor CDA

Added by Sven Vanpoucke about 13 years ago. Updated almost 12 years ago.

Status:
Feature implemented
Priority:
Immediate
Assignee:
Hans De Bisschop
Target version:
Start date:
27/01/2010
Due date:
% Done:

90%

Estimated time:

Description

Todo list

1 When adding new languages
a) Add new empty translation for all variabes
2 When deleting languages
a) Delete all related translations
3 When deleting a language pack
a) Delete all variables from language pack
b) Delete all translations of those variables
4 When creating a variable
a) Create an empty translation for all languages
5 When deleting a variable
a) Delete all related translations
6 Making a translation
7 Rating translations
8 Add moderators per language
9 Implement translation-request-form
10 Translation-request management (admin + moderator)
11 Implement view for individual translations
12 Implement locking functionality for translations
13 Implement zip-export for translations
14 Implement a progress bar
15 Add rating and source language to translation table
16 Support different branches
17 Checking for doubles among languages
18 Checking for doubles among language packs
19 Checking for doubles within language packs
20 Add a filter to show only untranslated variables
21 Add "next" mode to translation form to go from one variable to the next
22 Import existing language files (allows offline translating) (due to the language file format of 1.8, check the code to load sublanguages in 1.8)
23 (Re)Write a scan script to scan variables from chamilo 2.0
24 Create an advanced search form to search translations
25 Add rtl-ltr indicator in language table (Right-to-Left and Left-to-Right, would name it rtl and put it to 0 by default)
26 Add history to the translations


Files

english.zip (130 KB) english.zip Yannick Warnier, 05/02/2010 18:12
lang1.zip (601 KB) lang1.zip Yannick Warnier, 05/02/2010 20:00
lang2.zip (416 KB) lang2.zip Yannick Warnier, 05/02/2010 21:43

Related issues

Related to CDA - Feature #580: 'Dirty' translations detectionAssigned07/02/2010

Actions

History

#7

Updated by Sven Vanpoucke about 13 years ago

Added a zip exporter for both branch 1.x and 2.x in
http://code.google.com/p/chamilo/source/detail?r=64a6d796f8417bd26c3861a6532bee34e2fad322&repo=chamilo

Still needs some cleanup in layout

#9

Updated by Anonymous about 13 years ago

Blank page after selecting 'ALL' (or any other possibility but '20')when e.g. viewing (Dutch) translations in the ADMIN section (probably same problem in all sections);
blank page also when selecting next screen.

#10

Updated by Sven Vanpoucke about 13 years ago

Fixed this bug in
http://code.google.com/p/chamilo/source/detail?r=c63d531483a488fd899bc6b0cd77a6a72b855624&repo=chamilo

Checked the other tables as well but they didn't have that problem

#11

Updated by Sven Vanpoucke about 13 years ago

#13

Updated by Sven Vanpoucke about 13 years ago

Fixed sortable table bug and made it possible to sort on variables in translation browser
http://code.google.com/p/chamilo/source/detail?r=c075357782b96058451ee2254b2238e48532508a&repo=chamilo

#19

Updated by Sven Vanpoucke about 13 years ago

Added new todo list items

#21

Updated by Yannick Warnier about 13 years ago

Added one item for left-to-right and right-to-left (not urgent)

#22

Updated by Yannick Warnier about 13 years ago

Features start being alright now. The main worry now is the usability of the whole thing. It seems very difficult to me for a user to find his way to the translation.

  • A lot of untranslated terms in his way to the translation section
  • Asking to go to "Research -> CDA" to translate is like telling to look for a road sign in Russian to an extraterrestrial
  • "Help translating!" is alright
  • Choosing the language "from" and "to" is fine but it should default (the source) to the defined user language (but I believe there is no link between these, so leave it for now)
  • Adding variables seems to be an "admin-only" permission (am I wrong here?), which limits the system to only one or two people being able to add variables. Not sure this is right, but I'm OK with it for now
  • There should be a history of changes. At least one iteration, so that someone can "go back" to a previous translation of the same term
#23

Updated by Sven Vanpoucke about 13 years ago

Yannick Warnier wrote:

Features start being alright now. The main worry now is the usability of the whole thing. It seems very difficult to me for a user to find his way to the translation.

  • A lot of untranslated terms in his way to the translation section
  • Asking to go to "Research -> CDA" to translate is like telling to look for a road sign in Russian to an extraterrestrial
  • "Help translating!" is alright
  • Choosing the language "from" and "to" is fine but it should default (the source) to the defined user language (but I believe there is no link between these, so leave it for now)
  • Adding variables seems to be an "admin-only" permission (am I wrong here?), which limits the system to only one or two people being able to add variables. Not sure this is right, but I'm OK with it for now
  • There should be a history of changes. At least one iteration, so that someone can "go back" to a previous translation of the same term

I would like to answer to a few of your main worries/issues

  • The untranslated terms will not be a problem, once the tool is entirely finished, we run over our scan tool, put them in the cda, translate them directly at once and put them into the latest code, to avoid having untranslated variables.
  • The menu bar is completely configurable, meaning that we will make this a lot easier and remove loads of options from the bar for the end user to not be confused. We will only do this as a final step once we are sure we do not need to reinstall anymore because configuration would be lost anyway.
  • We can add a default value for the source translation to the user language if the language is available in the language list ofcourse
  • For now only an administrator can manage these variables. We could change it so a moderator can use it to, this is just a matter of putting some rights in the correct order
  • The only big issue now is the history, our architecture does currently not support an easy way to provide history.
#24

Updated by Hans De Bisschop about 13 years ago

Added one item for left-to-right and right-to-left (not urgent)

Makes sense to register that. Not completely unrelated: should we start using (perhaps additionally) the more extensive 3-letter ISO language codes? Examples of things which can't be described using default 2-letter codes are Brazilian Portuguese, Traditional vs. Simplified Chinese, (South) American Spanish, etc.

  1. A lot of untranslated terms in his way to the translation section
  2. Asking to go to "Research -> CDA" to translate is like telling to look for a road sign in Russian to an extraterrestrial

Those can actually be fixed rather simply ... stay tuned for the "solution"

Choosing the language "from" and "to" is fine but it should default (the source) to the defined user language (but I believe there is no link between these, so leave it for now)

No link = no problem ... it makes sense to select the user's default language.

Adding variables seems to be an "admin-only" permission (am I wrong here?), which limits the system to only one or two people being able to add variables. Not sure this is right, but I'm OK with it for now

Could be changed relative quickly by using the rights-system. All that needs to be implemented is a few extra checks here and there to make sure we keep out whoever shouldn't be in there.

There should be a history of changes. At least one iteration, so that someone can "go back" to a previous translation of the same term

And just when you think all's well that ends well ... there comes the tricky one ... requires some thought.

#28

Updated by Stefaan Vanbillemont almost 13 years ago

  • Tracker changed from Feature to To do
  • Project changed from Chamilo LCMS Connect to CDA
  • Category deleted (15)
  • Target version deleted (2)
#29

Updated by Hans De Bisschop almost 13 years ago

Fixed 26 in http://code.google.com/p/chamilo/source/detail?r=0c949c9c5a0a8411b47b2286d3143fec97e31e69&repo=chamilo

Some future extensions are still possible. They'll be added as seperate feature requests.

#30

Updated by Yannick Warnier almost 13 years ago

I'm having a hard time importing Arabic Unicode but I will get to it.
One thing that seems strange to me when I import everything directly to the database is that when importing them in order (arabic, asturian, ...) then I get no "source translation". Is there anything specific I need to indicate about English for it to be the source language? Should it be language "1" in all cases?

#31

Updated by Hans De Bisschop almost 13 years ago

I'm no expert on unicode and/or exotic languages and/or character sets / encodings, but could the encoding of the source file be the problem for the Arabic? Maybe we should ask Ivan for some advice concerning these kinds of things?

As far as the import / source language. English is the default, but (and there's always a but) as long as you haven't imported the english variables there won't be anything to display offcourse.

#32

Updated by Yannick Warnier almost 13 years ago

I should be OK to figure out the UTF-8 problem (not that bad myself in unicode related stuff).
My question about English is "Does it require to be language of ID 1 or is it OK for it to just be called "English" to be considered the default"?

#33

Updated by Yannick Warnier almost 13 years ago

When trying to import the 1.8 language pack with your import feature on a local CDA install (fresh from today), I get a lot of funny things...

First of all, the import of a 4.6M zip actually takes more than 10GB in logs on my system (I have a trace feature enabled by xdebug, so I guess there are i a huge lot of processing behind that).
Second, it uses more than 170MB of RAM, so it's getting above limits for my system (and crashes after 4 languages, so apparently importing only 4 languages took 10GB of trace)
Third, when I get into one of the resulting language packs (and click on the language pack name) in the normal CDA application (not admin), I get this (with xdebug enabled):

( ! ) Fatal error: Call to a member function get_id() on a non-object in /var/www/cda/application/lib/cda/tables/variable_translation_table/default_variable_translation_table_column_model.class.php on line 38
Call Stack
#    Time    Memory    Function    Location
1    0.0005    109596    {main}( )    ../run.php:0
2    0.5593    8454276    CdaManager->run( )    ../run.php:36
3    0.5730    9756536    CdaManagerVariableTranslationsBrowserComponent->run( )    ../cda_manager.class.php:196
4    0.8372    12420384    CdaManagerVariableTranslationsBrowserComponent->get_table( )    ../variable_translations_browser.class.php:41
5    0.8388    12424860    VariableTranslationBrowserTable->VariableTranslationBrowserTable( )    ../variable_translations_browser.class.php:51
6    0.8388    12425556    VariableTranslationBrowserTableColumnModel->VariableTranslationBrowserTableColumnModel( )    ../variable_translation_browser_table.class.php:25
7    0.8388    12425556    DefaultVariableTranslationTableColumnModel->DefaultVariableTranslationTableColumnModel( )    ../variable_translation_browser_table_column_model.class.php:28
8    0.8388    12425556    DefaultVariableTranslationTableColumnModel::get_default_columns( )    ../default_variable_translation_table_column_model.class.php:20

I'll try with one language at a time now.

#34

Updated by Yannick Warnier almost 13 years ago

Most of the space and memory usage is due to my tracing stuff apparently (much faster and less hungry without these features) but now I get:

( ! ) Fatal error: Call to a member function set_translation() on a non-object in /var/www/cda/application/lib/cda/cda_manager/component/translation_importer/translation_importer.class.php on line 169
Call Stack
#    Time    Memory    Function    Location
1    0.0002    113128    {main}( )    ../run.php:0
2    0.1286    8456328    CdaManager->run( )    ../run.php:36
3    0.1396    9799168    CdaManagerAdminTranslationImporterComponent->run( )    ../cda_manager.class.php:196
4    0.3074    11707084    TranslationImporter->import( )    ../admin_translation_importer.class.php:46
5    0.6970    12790188    TranslationImporter->import_translations( )    ../translation_importer.class.php:117

I'm attaching the English language import file (converted to utf-8) for tests on your side if you want

#35

Updated by Yannick Warnier almost 13 years ago

Added "&& is_object($system_translation) " as a temporary patch to line 167 of translation_importer.class.php and it imported English zip

#36

Updated by Hans De Bisschop almost 13 years ago

Having a look at the code now ... I'm rather curious as to what might possibly be going wrong. Thanks for adding the language pack you're using, makes things a lot easier.

#37

Updated by Yannick Warnier almost 13 years ago

Just tried to import all languages at once but that takes more than 500MB of RAM, so I'll try a few manually for now.
Memory usage is generally due to opening files in memory all at once (within the same loop but forgetting to cleaning between languages for examples)
500MB RAM allowed importing 10 languages (of which most are less than 50% translated), so I'll take a basis of 5 languages at a time for the zips

#38

Updated by Hans De Bisschop almost 13 years ago

System just finished importing your english.zip. No errors whatsoever ... :-S

#39

Updated by Yannick Warnier almost 13 years ago

Weird. Did you apply my hack?
Five languages in one ZIP passed over 500MB. I've assigned 1024M now and that seems to work.

#40

Updated by Yannick Warnier almost 13 years ago

Try this one (5 languages)

#41

Updated by Hans De Bisschop almost 13 years ago

No hack was applied. I did increase the memory limits before importing, but that shouldn't be related to the other error messages you were getting. Trying the 5 language-pack now.

#42

Updated by Hans De Bisschop almost 13 years ago

Import finished succesfully. Memory usage went just over 500MB.

#43

Updated by Yannick Warnier almost 13 years ago

Isn't that a lot for around 5600 terms per language? (=~100KB per term! That's like writing an entire essay on paper for each term, most of them being max 4 words)

#44

Updated by Hans De Bisschop almost 13 years ago

While I agree it's definately too much, by the end of the processing not 5600 but 5 x 5600 terms had been processed / created / updated. I'm curious as to what's causing the memory drain though.

#45

Updated by Yannick Warnier almost 13 years ago

Tried uploading lang2.zip attached (5 other languages) on translate.chamilo.org and got blank screen at the end. Did you try it there or on your computer?

#46

Updated by Yannick Warnier almost 13 years ago

OK, I have a bunch of requests for improvements, but I think we can start working on this. It still lacks a few little features to be as useful as the DLTT:

  1. split the 1.8 and 2.0 translations (it's just weird to have to translate something from a software you're not using, try to think as if you were a translator that wants to improve his language). This could be done by a simple filter inside the "save and go to next" cycle, just like the one on the language packs list...
  2. split the percentage between translations completion of 1.8 and 2.0 (it's just so weird to translate all the French language strings of 1.8 and still get 60%)
  3. somehow a few strings are there in English but with no translation at all and it's just wrong to have to translate something based on the variable without notion of what it actually means. Please make sure you always put the English translation in there
  4. somehow it is possible to select only one source language for one single user. Not sure why, but hey... seems strange to me (I can live with it, though - I guess it's difficult to guess from what language you want to translate if you translate from English and French to Spanish :)) -
  5. make sure the warning message when passing to the next term without translating the previous is seen as a warning, not an error (just feels guilty for not translating the previous one)
  6. in the "save and got to next" sequence, the new variable appearing should have an auto-focus on the translation box. Saves a lot of efforts when mass-translating
  7. when "skipping" a translation (like "My Research", for which I'm not sure what it means exactly), the system tends to repeat the skipped term over and over. It's boring and time consuming. I think it remains in the same "module" until it is finished...
  8. there should be more examples of translations than just the English one. When hesitating, seeing a translation in, say, Spanish, can help a lot!
  9. when confirming that a translation has been saved, also confirmed what it was (just in case that would let someone see he's made a typo)
OK, now I'm done criticizing :-)
The tool is really great as it is now. I recommend you have a look at phpLangEditor for Firefox (http://phplangeditor.mozdev.org/) in order to get an idea of how to speed translation even more. I would suggest putting a link to the extension somewhere visible into the CDA, so that people can help more easily.
Just so you don't loose too much time trying it out, you have to:
  • install the extension in Firefox (doesn't work with 3.6)
  • download a reference language pack (e.g. English) and a destination language pack (e.g. Dutch)
  • unzip the downloaded folder
  • open phpLangEditor
  • select "Open Reference file" and select the first file of the English directory
  • select "Open translation file" and select the first file of the Dutch directory
  • press CTRL+Enter each time you want to pass to the next step
  • save your file
  • zip the Dutch directory
  • import it to CDA
  • The End
#47

Updated by Hans De Bisschop almost 13 years ago

Split the 1.8 and 2.0 translations (it's just weird to have to translate something from a software you're not using, try to think as if you were a translator that wants to improve his language). This could be done by a simple filter inside the "save and go to next" cycle, just like the one on the language packs list...

Split the percentage between translations completion of 1.8 and 2.0 (it's just so weird to translate all the French language strings of 1.8 and still get 60%)

If the first one is implemented, the second one is a semi-natural result of it.

in the "save and got to next" sequence, the new variable appearing should have an auto-focus on the translation box. Saves a lot of efforts when mass-translating

Makes sense

there should be more examples of translations than just the English one. When hesitating, seeing a translation in, say, Spanish, can help a lot!

Somehow I just knew you were going to ask about that :p ... seeing as how the old translation-tool had that too. Not a problem though, I'll add an overview of all available translations in other languages to the translation page.

The skip-thing is something Sven implemented, but we discussed the problem you describe at length. (I think I was the one pointing out that it would be an issue. Not sure if he had already solved it or if it was on his to do-list)

The tool is really great as it is now. I recommend you have a look at phpLangEditor for Firefox (http://phplangeditor.mozdev.org/) in order to get an idea of how to speed translation even more. I would suggest putting a link to the extension somewhere visible into the CDA, so that people can help more easily.

Just so you don't loose too much time trying it out, you have to:

That tool is great for those of us who are too lazy to actually use the CDA itself. (I plead guilty) Combined with the import it makes life a whole lot easier.

#48

Updated by Sven Vanpoucke almost 13 years ago

Fixed a few small remarks in
http://code.google.com/p/chamilo/source/detail?r=a8a2d829233010c7b87b01a3803dc269c0d6fa9d&repo=chamilo

Nr 7 of yannick's comment's should be retested since i saw today that i did not commit every file i had.

#50

Updated by Anonymous almost 13 years ago

*Filter: when making use of the branch filter, the selected filter gets forgotten after nearly each 'action'
*When working from a long list of translations (e.g. for the repository in branch 2), the user is always redirected to the first screen after e.g. adding a translation on screen 5. Not very usable :-).

#51

Updated by Yannick Warnier almost 13 years ago

Hans De Bisschop wrote:

there should be more examples of translations than just the English one. When hesitating, seeing a translation in, say, Spanish, can help a lot!

Somehow I just knew you were going to ask about that :p ... seeing as how the old translation-tool had that too. Not a problem though, I'll add an overview of all available translations in other languages to the translation page.

Actually just showing it in English and Spanish should already fix about 90% of the real problems, most of the complete translations being related to the roots of those 3 languages (for now :-)).

The skip-thing is something Sven implemented, but we discussed the problem you describe at length. (I think I was the one pointing out that it would be an issue. Not sure if he had already solved it or if it was on his to do-list)

The tool is really great as it is now. I recommend you have a look at phpLangEditor for Firefox (http://phplangeditor.mozdev.org/) in order to get an idea of how to speed translation even more. I would suggest putting a link to the extension somewhere visible into the CDA, so that people can help more easily.

Just so you don't loose too much time trying it out, you have to:

That tool is great for those of us who are too lazy to actually use the CDA itself. (I plead guilty) Combined with the import it makes life a whole lot easier.

Actually for people in poor countries like us, it also avoids getting mad waiting for the page to load between 2 translations (it takes about 3-5 seconds between each). It is definitely a great tools combination anyway.

#52

Updated by Yannick Warnier almost 13 years ago

Would be nice to have Cda=SelectedTranslatorApplicationActivated translated :-)

#53

Updated by Yannick Warnier almost 13 years ago

I think we can close this task. It will be easier to track the others if we keep things neat and close the ones that have been completed (following two people's opinion)

#54

Updated by Stefaan Vanbillemont almost 13 years ago

  • Status changed from New to Feature implemented
#55

Updated by Stefaan Vanbillemont almost 12 years ago

  • Target version set to 2.1.0

Also available in: Atom PDF