Project

General

Profile

Feature #272

UTF-8 native support

Added by Carlos Vargas about 10 years ago. Updated over 7 years ago.

Status:
Feature implemented
Priority:
Normal
Category:
-
Target version:
Start date:
03/12/2009
Due date:
% Done:

100%

Estimated time:
Complexity:
Normal
SCRUM pts - complexity:
100

Description

An e-mail from cristi hypermedia.eu

---
Hello,

We’re sorry for the delay, but we’ve been pretty busy here at Hypermedia :)
I could send you the whole version that I’ve modified from Dokeos 1.6.3 , but there are many modifications – customizations in fact that were needed at that moment.
(if you want the whole version I modified, just say so and it will be no problem for me to send it. I just think that it will be way to complicated to follow all the modifications, because I cannot remember all the things I’ve changed :D).
I mean no offense if there are too many explanations, but I wrote the document this way, so that maybe we can use it to show a “non-technical” person how to migrate.

Here is the roadmap I followed (actually I wrote it afterwards) for UTF8 conversion of Dokeos:
I’ve used here the MySql command line because it’s faster. If you want to use phpmyadmin, I think you should be careful what ‘MySQL charset’ and what ‘MySQL connection collation’ are you using.
I will try this sometime, but for the smoothest results my guess is that:
- before making the dump you have to use MySQL charset latin1 and MySQL connection collation latin1_swedish (the default for the Mysql tables)
- before loading the dump (converted to UTF8) you have to change MySQL charset to utf8 and MySQL connection collation utf8_general_ci
It might work in any case, but I think this is the easiest way.

1. Database:
Currently the whole Dokeos database uses latin1_swedish charset in MySql.
There are a few ways of converting the whole database. Many say to convert all your ‘text’ fields (varchar included) to binary and then run queries.
But I have this way of doing it, which is much more “nice” and VERY MUCH FASTER.

- Create dump with latin1 charset:
mysqldump --default-character-set=latin1 u root -p dokeos_main > dokeos_latin1.sql

Remove (or replace) all the charset-related directives. For example – you could replace ‘DEFAULT CHARSET=latin1’ with ‘DEFAULT CHARSET=utf8’ in all the document.
Or, if you create your new database with default collation utf_general_ci , you could just remove all the ‘DEFAULT CHARSET=latin1’ from the file, and then when you import it, MySql will use the database default charset for all the tables.

- Convert the dump file from latin1 (iso-8859-1) to utf-8
You could use several ways of doing this, but the nicest I have so far is by using the iconv library (on Linux is installed by default I guess, for Windows I downloaded it from http://gnuwin32.sourceforge.net/packages/libiconv.htm)
iconv f iso-8859-7 -t utf-8 dokeos_latin1.sql > dokeos_utf8.sql

Create a new database with collation utf8_general_ci – database dokeos_utf8.sql

- import the new file with utf-8 charset in the new database
mysql --default-character-set=utf8 u root -p dokeos_utf < dokeos_utf8.sql

Check with phpmyadmin using utf connection and collation to see if texts from all languages are ok (greek is just an example because it uses non latin characters and that’s what we’ve been using here in Cyprus).

2. Language files convert from iso-8859-1 to utf-8:
I used again the iconv library. For the greek files I used :
iconv.exe -f iso-8859-7 -t utf-8 language_file.inc.php > utf/language_file.inc.php

all the language files in claroline/lang
- problems in chat.inc.php - remove special characters or convert to html entities (I have to clear what was this about sometime) – it seems there are some greek special characters that are not “fully” supported for conversion.

Additional language conversions:

- ctools/wiki/greek.inc.php

- claroline/ctools/glossary/lang/greek/glossarie.inc.php

- egnosis/home/home_*_greek.html - files with texts from home page

- ctools/poll/calendar/popcalendar.js - greek months names

I did these conversion because we used those plugins and tolls in the platform.

3. Modify code-files in claroline directory:
Notice that line numbers might not be the same – I told you I made many modifications on the platform.

There is a catch here.

Search for all mysql_connect() calls and add SET NAMES and SET CHARACTER SET
- inc\\claro_init_global_inc.php - added lines 64,65 (after mysql_select_db())
mysql_query("SET NAMES 'utf8'", $dokeos_database_connection);
mysql_query("SET CHARACTER SET 'utf8'", $dokeos_database_connection);

- ctools\\glossary\\glossary.php
line 291 - added function utf8_substr() and used it everywhere in the document instead of substr()

- plugin/CoolSearch/ - implemented separate language files for English and Greek using UTF8

- announcements/announcements.php - line 619: UTF-8 encoding for email (instead of iso-8859-1)

- admin/user_import.php - UTF-8 charset for the imported file (?!)
- admin/user_export.php - UTF-8 charset for the exported file

- auth/inscription_second.php - utf-8 charset for the email sent to the new user

- calendar/calendar.php - charset of the calendar page

- inc/claro_init_header.inc.php - set default charset to UTF-8

- inc/phpmailer/class.phpmailer.php - line 36: set utf-8 encoding for mails
- inc/phpmailer/phpdoc/phpmailer.html - description of phpmailer class - line 526: default encoding of mails is utf-8
- inc/phpmailer/test/phpmailer_test.php - lines 63,198: set mail's encoding to utf-8

- lang/english/trad4all.inc.php - line 18: $charset='utf-8';
- lang/(english|greek)/tracking.inc.php - add $status_lang - array with possible status of a lesson

- learnpath/learnpath_functions.inc.php - line 2301: set charset of the manifest file to utf-8

- online/header_frame.inc.php - line 67: set page encoding to utf-8
- online/online.php - line 46: set page encoding to utf-8

- scorm/closesco.php - line 49: set page encoding to utf-8
- scorm/contents.php - line 68: set page encoding to utf-8
- scorm/opensco.php - line 51: set page encoding to utf-8

- tracking/courseLog.php
- line 60: set page encoding to utf-8
- line 108: if $status_lang not imported from language file, put it in english language
- tracking/userLog.php
- lines 81,85: set default translations encoding to utf-8
- line 67: if $status_lang not imported from language file, put it in english language

- work/student.html - line 4: set page encoding to utf-8

Also, there are some characters that are not displayed correctly in utf8

- CHANGE special characters that are displayed garbled in UTF8 to HTML entities:
- « and » instead of « and »
- calendar/agenda.inc.php
- calendar/myagenda.php

- CHANGE CREATE DATABASE syntax - DEFAULT CHARACTER SET utf8 for all tables created from within php scripts:
- claroline/calendar/myagenda.php - line 113
- claroline/inc/lib/add_course.lib.inc.php - line 199
- claroline/install/install_db.inc.php - line 127, 143, 161, 179 - leave CREATE TABLE syntax as it is for now

- CHANGE CREATE TABLE syntax DEFAULT CHARSET = utf8:
- claroline/inc/lib/add_course.lib.inc.php

ctools/poll/calendar/popcalendar.js - months names
- phpbb/reply.php - line 526: replaced htmlentities() function with htmlspecialchars()

Here it is now a list of the PROBLEMATIC PHP functions when working with UTF8 strings:
- String functions - they all have equivalents in mbstring extension (prefixed with 'mb_') :
- strcut()
- strlen()
- strpos()
- strrpos()
- strtolower()
- strtoupper()
- substr_count()
- substr() I used built-up utf8_substr() function for now

The best solution to fix these problem is to test if the mbstring extension for php is installed and work with equivalent string functions if yes. If it is not installed, we could use the iconv() functions from PHP to convert strings on the fly, but of course this will slow down things. However there might be many cases when you cannot use mbstring evan if you want to (hosting companies).
In Dokeos I used :

function utf8_substr($str,$from,$len){
return preg_replace('#^(?:[\\x00-\\x7F]|[\\xC0-\\xFF][\\x80-\\xBF]+){0,'.$from.'}'.
'((?:[\\x00-\\x7F]|[\\xC0-\\xFF][\\x80-\\xBF]+){0,'.$len.'}).*#s',
'$1',$str);
}
And that pretty much solved everything – because the rest of the functions are not used in a “harmful” way for utf8 strings.

value of mbstring.func_overload - original function overloaded function
1 - mail() - mb_send_mail()
2 - strlen() - mb_strlen()
2 - strpos() - mb_strpos()
2 - strrpos() - mb_strrpos()
2 - substr() - mb_substr()
2 - strtolower() - mb_strtolower()
2 - strtoupper() - mb_strtoupper()
2 - substr_count() - mb_substr_count()
4 - ereg() mb_ereg()
4 - eregi() - mb_eregi()
4 - ereg_replace() - mb_ereg_replace()
4 - eregi_replace() - mb_eregi_replace()
4 - split() - mb_split()

I think this is all. If I remember something else I will send other emails.
Also, if you have any questions, or if you want me to help in this direction, I could install a CVS system and grab the latest files from the repository and convert all the language files, database creation files etc. etc.
Just let me know what are your plans and we can decide together how we can help each other :)

Awaiting to hear fAn e-mail from cristi hypermedia.eu

---
Hello,

We’re sorry for the delay, but we’ve been pretty busy here at Hypermedia :)
I could send you the whole version that I’ve modified from Dokeos 1.6.3 , but there are many modifications – customizations in fact that were needed at that moment.
(if you want the whole version I modified, just say so and it will be no problem for me to send it. I just think that it will be way to complicated to follow all the modifications, because I cannot remember all the things I’ve changed :D).
I mean no offense if there are too many explanations, but I wrote the document this way, so that maybe we can use it to show a “non-technical” person how to migrate.

Here is the roadmap I followed (actually I wrote it afterwards) for UTF8 conversion of Dokeos:
I’ve used here the MySql command line because it’s faster. If you want to use phpmyadmin, I think you should be careful what ‘MySQL charset’ and what ‘MySQL connection collation’ are you using.
I will try this sometime, but for the smoothest results my guess is that:
- before making the dump you have to use MySQL charset latin1 and MySQL connection collation latin1_swedish (the default for the Mysql tables)
- before loading the dump (converted to UTF8) you have to change MySQL charset to utf8 and MySQL connection collation utf8_general_ci
It might work in any case, but I think this is the easiest way.

1. Database:
Currently the whole Dokeos database uses latin1_swedish charset in MySql.
There are a few ways of converting the whole database. Many say to convert all your ‘text’ fields (varchar included) to binary and then run queries.
But I have this way of doing it, which is much more “nice” and VERY MUCH FASTER.

- Create dump with latin1 charset:
mysqldump --default-character-set=latin1 u root -p dokeos_main > dokeos_latin1.sql

Remove (or replace) all the charset-related directives. For example – you could replace ‘DEFAULT CHARSET=latin1’ with ‘DEFAULT CHARSET=utf8’ in all the document.
Or, if you create your new database with default collation utf_general_ci , you could just remove all the ‘DEFAULT CHARSET=latin1’ from the file, and then when you import it, MySql will use the database default charset for all the tables.

- Convert the dump file from latin1 (iso-8859-1) to utf-8
You could use several ways of doing this, but the nicest I have so far is by using the iconv library (on Linux is installed by default I guess, for Windows I downloaded it from http://gnuwin32.sourceforge.net/packages/libiconv.htm)
iconv f iso-8859-7 -t utf-8 dokeos_latin1.sql > dokeos_utf8.sql

Create a new database with collation utf8_general_ci – database dokeos_utf8.sql

- import the new file with utf-8 charset in the new database
mysql --default-character-set=utf8 u root -p dokeos_utf < dokeos_utf8.sql

Check with phpmyadmin using utf connection and collation to see if texts from all languages are ok (greek is just an example because it uses non latin characters and that’s what we’ve been using here in Cyprus).

2. Language files convert from iso-8859-1 to utf-8:
I used again the iconv library. For the greek files I used :
iconv.exe -f iso-8859-7 -t utf-8 language_file.inc.php > utf/language_file.inc.php

all the language files in claroline/lang
- problems in chat.inc.php - remove special characters or convert to html entities (I have to clear what was this about sometime) – it seems there are some greek special characters that are not “fully” supported for conversion.

Additional language conversions:

- ctools/wiki/greek.inc.php

- claroline/ctools/glossary/lang/greek/glossarie.inc.php

- egnosis/home/home_*_greek.html - files with texts from home page

- ctools/poll/calendar/popcalendar.js - greek months names

I did these conversion because we used those plugins and tolls in the platform.

3. Modify code-files in claroline directory:
Notice that line numbers might not be the same – I told you I made many modifications on the platform.

There is a catch here.

Search for all mysql_connect() calls and add SET NAMES and SET CHARACTER SET
- inc\\claro_init_global_inc.php - added lines 64,65 (after mysql_select_db())
mysql_query("SET NAMES 'utf8'", $dokeos_database_connection);
mysql_query("SET CHARACTER SET 'utf8'", $dokeos_database_connection);

- ctools\\glossary\\glossary.php
line 291 - added function utf8_substr() and used it everywhere in the document instead of substr()

- plugin/CoolSearch/ - implemented separate language files for English and Greek using UTF8

- announcements/announcements.php - line 619: UTF-8 encoding for email (instead of iso-8859-1)

- admin/user_import.php - UTF-8 charset for the imported file (?!)
- admin/user_export.php - UTF-8 charset for the exported file

- auth/inscription_second.php - utf-8 charset for the email sent to the new user

- calendar/calendar.php - charset of the calendar page

- inc/claro_init_header.inc.php - set default charset to UTF-8

- inc/phpmailer/class.phpmailer.php - line 36: set utf-8 encoding for mails
- inc/phpmailer/phpdoc/phpmailer.html - description of phpmailer class - line 526: default encoding of mails is utf-8
- inc/phpmailer/test/phpmailer_test.php - lines 63,198: set mail's encoding to utf-8

- lang/english/trad4all.inc.php - line 18: $charset='utf-8';
- lang/(english|greek)/tracking.inc.php - add $status_lang - array with possible status of a lesson

- learnpath/learnpath_functions.inc.php - line 2301: set charset of the manifest file to utf-8

- online/header_frame.inc.php - line 67: set page encoding to utf-8
- online/online.php - line 46: set page encoding to utf-8

- scorm/closesco.php - line 49: set page encoding to utf-8
- scorm/contents.php - line 68: set page encoding to utf-8
- scorm/opensco.php - line 51: set page encoding to utf-8

- tracking/courseLog.php
- line 60: set page encoding to utf-8
- line 108: if $status_lang not imported from language file, put it in english language
- tracking/userLog.php
- lines 81,85: set default translations encoding to utf-8
- line 67: if $status_lang not imported from language file, put it in english language

- work/student.html - line 4: set page encoding to utf-8

Also, there are some characters that are not displayed correctly in utf8

- CHANGE special characters that are displayed garbled in UTF8 to HTML entities:
- « and » instead of « and »
- calendar/agenda.inc.php
- calendar/myagenda.php

- CHANGE CREATE DATABASE syntax - DEFAULT CHARACTER SET utf8 for all tables created from within php scripts:
- claroline/calendar/myagenda.php - line 113
- claroline/inc/lib/add_course.lib.inc.php - line 199
- claroline/install/install_db.inc.php - line 127, 143, 161, 179 - leave CREATE TABLE syntax as it is for now

- CHANGE CREATE TABLE syntax DEFAULT CHARSET = utf8:
- claroline/inc/lib/add_course.lib.inc.php

ctools/poll/calendar/popcalendar.js - months names
- phpbb/reply.php - line 526: replaced htmlentities() function with htmlspecialchars()

Here it is now a list of the PROBLEMATIC PHP functions when working with UTF8 strings:
- String functions - they all have equivalents in mbstring extension (prefixed with 'mb_') :
- strcut()
- strlen()
- strpos()
- strrpos()
- strtolower()
- strtoupper()
- substr_count()
- substr() I used built-up utf8_substr() function for now

The best solution to fix these problem is to test if the mbstring extension for php is installed and work with equivalent string functions if yes. If it is not installed, we could use the iconv() functions from PHP to convert strings on the fly, but of course this will slow down things. However there might be many cases when you cannot use mbstring evan if you want to (hosting companies).
In Dokeos I used :

function utf8_substr($str,$from,$len){
return preg_replace('#^(?:[\\x00-\\x7F]|[\\xC0-\\xFF][\\x80-\\xBF]+){0,'.$from.'}'.
'((?:[\\x00-\\x7F]|[\\xC0-\\xFF][\\x80-\\xBF]+){0,'.$len.'}).*#s',
'$1',$str);
}
And that pretty much solved everything – because the rest of the functions are not used in a “harmful” way for utf8 strings.

value of mbstring.func_overload - original function overloaded function
1 - mail() - mb_send_mail()
2 - strlen() - mb_strlen()
2 - strpos() - mb_strpos()
2 - strrpos() - mb_strrpos()
2 - substr() - mb_substr()
2 - strtolower() - mb_strtolower()
2 - strtoupper() - mb_strtoupper()
2 - substr_count() - mb_substr_count()
4 - ereg() mb_ereg()
4 - eregi() - mb_eregi()
4 - ereg_replace() - mb_ereg_replace()
4 - eregi_replace() - mb_eregi_replace()
4 - split() - mb_split()

I think this is all. If I remember something else I will send other emails.
Also, if you have any questions, or if you want me to help in this direction, I could install a CVS system and grab the latest files from the repository and convert all the language files, database creation files etc. etc.
Just let me know what are your plans and we can decide together how we can help each other :)

Awaiting to hear from you…

Best Regards,
Cristi Ionescu
rom you…

Best Regards,
Cristi Ionescu


Files

database_encoding_scheme.png (12.2 KB) database_encoding_scheme.png Ivan Tcholakov, 18/01/2010 09:21
wiki-mpdf-export-utf-8-bulgarian.png (60.7 KB) wiki-mpdf-export-utf-8-bulgarian.png Ivan Tcholakov, 28/01/2010 02:21
sample_competences_report.png (70.4 KB) sample_competences_report.png Ivan Tcholakov, 30/01/2010 22:41
20100210_chamilo_database_migration_scheme.odg (15.6 KB) 20100210_chamilo_database_migration_scheme.odg Ivan Tcholakov, 10/02/2010 22:22
20100210_chamilo_database_migration_scheme.png (28.7 KB) 20100210_chamilo_database_migration_scheme.png Ivan Tcholakov, 10/02/2010 22:22
strange_duplication.png (54.6 KB) strange_duplication.png Ivan Tcholakov, 04/03/2010 01:32
dates_encoding_problem.png (101 KB) dates_encoding_problem.png Ivan Tcholakov, 07/03/2010 02:04

Related issues

Related to Chamilo LMS - Bug #551: Make more language packs available in UTF-8Bug resolved23/01/201015/02/2010

Actions
Related to Chamilo LMS - Support #621: Rework the gradebook PDF export to use mPDFBug resolved19/02/2010

Actions
Related to CDA - Bug #627: Some problems about Chamilo 1.8.x translationFeature implemented22/02/2010

Actions
Related to Chamilo LMS - Feature #542: Change of name for some files and database namesFeature implemented20/01/2010

Actions
Related to Chamilo LMS - Bug #605: get_lang(): Translation suggestion links have to point to Chamilo siteFeature implemented12/02/2010

Actions
Related to Chamilo LMS - Bug #696: The function api_get_local_time() has problematic designBug resolved10/03/2010

Actions
Related to Chamilo LMS - Bug #761: Hide and show not workingFeature implemented17/03/2010

Actions
Related to Chamilo LMS - Bug #825: Illegal mix of collations (utf8_spanish_ci, IMPLICIT) and (utf8_general_ci, IMPLICIT)Feature implemented23/03/2010

Actions
Related to Chamilo LMS - Bug #1713: Duplicate Query in Update db 1.8.6.2 to 1.8.7Feature implemented10/07/2010

Actions
Related to Chamilo LMS - Bug #1734: Encoding problem in "news" (Chinese characters don't show)Feature implemented15/07/2010

Actions

History

#1

Updated by Carlos Vargas about 10 years ago

Actualizado por Ludovic Gasc hace más de 3 años

Comment

It's very interessant, but much details must be tested to be sure we don't have any problems :
- FCKeditor areas
- File uploads
- strings and pages in Learning path
- databases
- e-mails

Moreover, mbstring http://fr.php.net/manual/en/ref.mbstring.php and iconv http://fr.php.net/manual/en/ref.iconv.php isn't by default in php, it's necessary to keep old system in dokeos in case of a php install without theses extensions.

For which version of dokeos does have one to set up to utf-8 ? I think it's too short for dokeos 1.8.
#2
Actualizado por Ludovic Gasc hace más de 3 años

Comment

This is tar.gz with utf-8 version and comments from e-mail :
-----
Here is the utf8 version of dokeos I've built.
I've sent only the claroline directory and the index.php. The rest of the files in the root directory are unchanged. I replaced all the 'dokeos' string with 'egnosis' - that's how we named the platform.
In the claroline/lang folder you will find only the English and greek folders, because this is only what we used.
Also I have removed the claroline/help directory, because we made a static version of the help and it has 10 MB. So don't be surprised if the help links don't work.

If I can assist you with anything else, send an email please.
#3
Actualizado por Yannick Warnier hace alrededor de 1 año

Comment

Language files have progressively been enabled as UTF-8 (for those who were relying on HTML characters). However, this article remains super-interesting as a preparation for 1.8.6 or 1.8.7 full-UTF-8 mode.
#4
Actualizado por Yannick Warnier hace alrededor de 1 año

Comment

http://www.dokeos.com/forum/viewtopic.php?p=45183#45183
#5
Actualizado por Ivan Tcholakov hace alrededor de 1 año

  • Fichero 20080904_dokeos_main_api_svn_16235.zip añadido

Comment

Hello.

I am working on a customized version of Dokeos using the current code form the svn-repository. Now the code there is between Dokeos 1.8.5 and Dokeos 1.8.6.

My goals are achieving better support for bulgarian language and UTF-8 encoding, providing some additional functionality (not important here) and making some cosmetics. Here I want to share my experience concerning UTF-8.

1.

I rely on mbstring extension for PHP 5, anyway it is now a madatory requirement for installation. But..., I do not use function overloading, because this would be another issue that clients would have to take care about during the installation. And I keep in mind that function overloadind does not solve all the problems authomatiaclly, but causes loosing of precise control from a programmer's point of view.

So I have chosen to browse all the code and manually to alter string functions calls: strlen(...) becomes mb_strlen(...), substr(...) becomes mb_substr(...), etc. With Eclipse-PDT IDE this is easy.

2.

I have realized that for utf-8 some specialized functions have to be implemented and their natural place is within the file main_api.lib.php. At the moment these functions are placed in a separate file main_api_extension.lib.php which is included ata the beginning of main_api.lib.php. Here I am attaching a zip-archive with these two files.

3.

A bunch of code is not enough, so I will try to explain or justify the decisions hidden inside.

Some "mb_"-like functions are written in PHP, because they are not always exist. See the following:
- The functions mb_strchr(), mb_stristr(), mb_strrchr(), mb_strstr() are implememented since PHP 5.2.0.
- The functions mb_str_ireplace(), mb_strcasecmp(), mb_strrev(), mb_substr_replace(), mb_str_split(), mb_ucfirst(), mb_lcfirst(), mb_ucwords(), mb_strtr() are not implemented at all. And this list may increase.
This is why using function overloding is not an option.

There are some functions to deal with single characters within a string:
api_first_character(...), api_last_character(...), api_remove_last_character(...);
api_character($string, $index, $encoding = null) - this one selects a character by its position (0 means the first character). For UTF-8 strings it is wrong way to select a character treating the given string as one-dimensional array;
api_characters() - this function transforms a string into an array of characters. It is convenient to be used in several places.

And there are some utf-8 aware functions that are close to the system Dokeos:

api_is_utf8($encoding = null) - returns true if the input string is "UTF-8", "utf-8", "Utf-8"..., othrwise it returns false. For example this function is usefull to check whether UTF-8 encoding is declared in a document's metadata;

api_set_mb_internal_encoding($encoding) - sets internal encodings for common mbstring function and mb_eregi functions. I prefer to keep these two encodungs synchronized.

api_migrate_to_utf8($string, $from_encoding = null, $language = null) - When the global variable $charset is "UTF-8", this function checks whether the given string is utf-8 valid one. If the string is not utf-8 valid, it is converted to utf-8 from the provided non-utf-8 encoding. If $encoding is null, then a correspondent to the given language non-utf-8 encoding is used. If $language is null also then a correspondent to the global variable $language_interface non-utf-8 encoding is used.
For example: this function is convenient to be used when CSV-files are imported and some of them utf-8 encoded but other ones are exported from "old" system Dokeos and have language-specific encoding. The same case is when some "old" course archives are restored and ater unserialization the strings within course resources possibly are not utf-8 encoded.
Another example: See <a href="http://www.dokeos.com/forum/viewtopic.php?t=11438&highlight=restorer" target="_blank">Importing (partly) old course_backup (1.6) to Dokeos 1.8</a>.I have tried Mark's partial solution to import my windows-1251 encoded course archives created by Dokeos 2.0.4 CR. But first I had extended the course and resource classes to use api_migrate_to_utf8() and thus the conversion has been done transparently.

api_add_unicode_modifier($re, $encoding = null) - When we deal with perl-compatible regular expressions to process utf-8 strings, a so called unicode modifier (letter "u") has to be added at the end of the expression string. This function is intended to do so.

api_get_language_specific_encoding($language = null) - This function returns correspondent to a given language non utf-8 encoding. For example when language is "bulgarian" the result is "windows-1251". During the conversion to utf-8 this function helps to determine what is the "legacy" encoding.

api_is_valid_utf8($string) - This function checks whether a given string is valid utf-8 one. See <a href="http://php.net/manual/en/function.mb-detect-encoding.php#48955" target="_blank">http://php.net/manual/en/function.mb-detect-encoding.php#48955&lt;/a&gt;. mb_detect_encoding() does not seem to be reliable. The solution from <a href="http://w3.org/International/questions/qa-forms-utf-8.html" target="_blank">http://w3.org/International/questions/qa-forms-utf-8.html&lt;/a> and its derivates are not reliable according to my personal experience.

Finally there are two MySQL-specific functions to deal with database encodings and collations.

api_get_database_charset_string($encoding = null) - This function translates to MySQL-specific notation from a given standard notation of encoding. To be clear: "utf-8" becomes "utf8", "iso-8859-1" becomes "latin1", "windows-1251" becomes "cp1251", "iso-8859-7" becomes "greek", etc. For example, this function is needed to determine the default character set for newly created created tables, to do this the Dokeos system encoding must be set (or known).
Maybe the reverse function also will be needed, when we upgrading an old system where the tables are not utf-8 encoded.

api_get_database_collation_string($encoding = null, $language = null) - When a new table is created it is better to set its collation too. This function does this.
Examples:
When $encoding="utf-8" and $language="bulgarian" the function returns "utf8_unicode_ci".
When $encoding="windows-1251" and $language="bulgarian" the function returns "cp1251_bulgarian_ci".

Some general rules for all functions:
---------------------------------------------------------------------------------------------------------------
When $encoding input variable is null or missing, the returned form mb_internal_encoding() value is used.
When $language input variable is null or missing, the value of the global variable $language_interface is used.

4.

And there are some other issues I would like to mention.

I have chosen the approach to alter all the string functions in Dokeos (except those in the libraries) with their correspondent "mb_string" functions. Even if they process file system paths or urls. Otherwise I would have to analyze too much code. There are no noticed problems so far.

The functions for avoiding "dangerous" characters or accent characters maybe have to be reworked. For bulgarian language I am thinking about transliteration when file names have to be created, i.e. replacing cyrillic letters with their similar laninian letters.

Concerning the PDF-export: In thus modified system ezpdf does not work, Mark's proprsal to use tcpdf works. But his solution is not finished.
These pdf-libraries seem cryptic to me. I can not help here at the moment.

Best regards,
Ivan Tcholakov
#6
Actualizado por Ivan Tcholakov hace alrededor de 1 año

  • Fichero Screenshot.png añadido

Comment

Hello again,

Please,

Avoid using these functions: utf8_encode() utf8_decode(). They support conversions only from/to ISO-8859-1 and UTF-8 encodings.
When conversions are made from/to different then ISO-8859-1 language specific encodings, the produced strings are not readable.
Look at the attached picture to see how utf8_decode() works for Bulgarian language and single-byte encoding windows-1251.

A better option is to use mb_convert_encoding().

Cheers,
Ivan Tcholakov
#7
Actualizado por Ivan Tcholakov hace alrededor de 1 año

  • Fichero Screenshot.png añadido

Comment

Hello,

I would like to focus your attention on "platform_charset" configuration setting in the administrator's panel (see the picture).

Currently, during the instalation of the system Dokeos, the character set of MySQL databases is "latin1" which is hard-coded. The installation script does not ask about encodings at all.

When the platform administrator changes the character set through its interface, maybe the yet created courses will not be shown correctly. But there is another thing for the newly created courses: possiblly the system seemingly will work correctly, but the text within the databases will be wrongly encoded. And when you try to see the text fields using phpMyAdmin or some other database tools you will see mess. When you try to export thus created courses and import them to another system that is configured correctly - nothing good will happen. And when you try to upgrade and migrate to utf-8 the whole system ....

There is no mechanism yet to make equal the encodings of the database tables and the web pages. I have an idea (not tested yet) how this mechanism should work:

1. During the installation all the databases will be created with utf8 character set. Thus the databases will be able to store all the corses in all their languages.

2. If there is a reason forsing the platform administrator to choose different than utf-8 encoding, the database server will continue to store the strings in utf-8, but it will communicate with the php5-client using the given encoding. I think that MySQL server provides this functionality.

Additionally I think that "platform_charset" setting should not affect the encoding of the exported XML or CSV files. It is better if they allways be utf-8 encoded. I have no time to check the behavior of the system about this.

All these writings so far were to justify a suggestion of mine that concerns the present situation. Please, make the setting "platform_charset" to be not editable in the "Production Server" mode. This will prevent possible incidents leading to data corruption.

Regards,
Ivan Tcholakov
#8
Actualizado por Yannick Warnier hace alrededor de 1 año

Comment

Hi Ivan,

Concerning this specific aspect, we do not care (intentionally) about the database encoding because, if Dokeos is the only interface to input and output data to and from MySQL, then it won't matter, as the database keeps whichever data in the encoding that fits best. It is true that accessing with phpMyAdmin will show differences in encoding, but this is a much minor concern to us than risking that the 5000 Dokeos installed around the world break on upgrade because we selfishly decided it would be cleaner to consider the databases to be encoded in UTF-8/unicode.

Furthermore, you can switch the encoding of your phpMyAdmin to make it consider you are viewing UTF-8 data.
#9
Actualizado por Ivan Tcholakov hace alrededor de 1 año

Comment

You are right about the number 5000.

Well, database encoding is not to be changed during upgrades. For installed yet systems nothing will change.

Imagine, when you install the system, it will ask you to select the language, and below there
will be another select box, "Choose database encoding/character set" ---> latin1 or utf8 .

For every detail the user will make a choice. The developers do not decide anything, they are just giving
more new possibilities.

I want to inform you that I am working on a plugin for full restoration of old course archives. I have
"borrowed" the idea and the initial work from mark111. The plugin restores learning paths too. And: for those
with $charset UTF-8 the plugin proposes to convert the whole course (database strings, some types of files).
In my estimation, I need one more week and then the plugin will be ready to be published and to be tried by
other people.

Has UTF-8 any chance to happen in Dokeos 1.8.7?

Cheers.
#10
Actualizado por Yannick Warnier hace alrededor de 1 año

Comment

http://dokeoslead.wordpress.com/2008/10/05/mbstring-vs-iconv-benchmarking/
#11
Actualizado por Yannick Warnier hace alrededor de 1 año

Comment

I have added this task as one of the objectives for 1.8.6. This might be changed during the process, but I think we should at least try and improve the code considerably, using all the recommendations we now have here.
#12
Actualizado por Ivan Tcholakov hace alrededor de 1 año

  • Fichero main_api_extensions.zip añadido

Comment

I read the article from http://dokeoslead.wordpress.com/2008/10/05/mbstring-vs-iconv-benchmarking/
some time ago. This conclusion appears:

mbstring --> good preformance
iconv --> large number of supported encodings
Why wouldn't we exploit them both?

I am attaching the current and "frozen" extra-API that demonstrates this approach.
The files global.inc.php and main_api.lib.php are the current svn versions with
modifications only for integrating the mylibyte-enabled API-extension.

How to use the multibyte string functions:
strlen($string) -----> api_strlen($string)
etc.

I am using/testing this API on a modified UTF-8 system in Bulgarian language
and I am personally satisfied.

The previous code that I've published here now looks to me naive. The current
code is mature enough.

License: GNU General Public License as published by the Free Software Foundation
version 2 or any later version

With best regards.
#13
Actualizado por Yannick Warnier hace alrededor de 1 año

Comment

Hi Ivan,

Please don't forget that we always want to have the minimum possible pre-requisites to install Dokeos and, as far as I seem to remember, iconv is an additional extension to the default installed on most systems by PHP5.2+. As such, if mbstring is faster and supports the encodings we need, I really wouldn't bother with iconv. However, you might have another take on this. Is the Bulgarian encoding not supported by mbstring?
#14
Actualizado por Ivan Tcholakov hace alrededor de 1 año

Comment

Here WINDOWS-1251 encoding is sometimes used. I don't need iconv.
But since the solution has to be common for all, the needs of
the "minority" of the community should be taken into account.

WINDOWS-1250 seems to be the biggest "minority". This encoding
in not supported by mbstring neither by htmlentities().
The languages that use this encoding seem to be:
bosnian
croatian
czech
polish (?)
slovak
slovenian

Mark111 is sadder but wiser about this. His opinion is more
valuable than mine. Please, look here:
http://www.dokeos.com/forum/viewtopic.php?t=12009

And more indicative posts:
http://www.dokeos.com/forum/viewtopic.php?t=5095
http://www.dokeos.com/forum/viewtopic.php?t=13464

Even the system moves to UTF-8, old and rare encodings to convert form
still have to be used when the system is upgraded, or when old courses
archives are restored, or... when importing SCORM packages with
non-UTF-8 encodings (the last is hypothetical).

This API first checks if a given encoding is supported by mbstring.
If this check is negative, then the API checks for iconv presense.
For the "majority" of the community iconv simply will not be needed.

Something offtopic concerning iconv: I searched globally the Dokeos
code for the word "iconv" and there were some interesting hits. Looks
like we are starting to depend on it anyway.

After two or three days I will discretly publish here the old courses
archives convertor that uses a copy of this API. The functions there
are declared conditionaly in order to avoid name conflicts, i.e.:

function api_strlen() { }

becomes

if (!function_exists('api_strlen')) {
function api_strlen() { }
}

I don't insist on anything. This work is what I need and I will
exploit it. And I am just sharing it here.

With best regards.
Ivan Tcholakov
#15
Actualizado por Yannick Warnier hace alrededor de 1 año

Comment

Thanks Ivan, we are all trying to make it better in some way and I'll try to use your code for the benefit of all. I didn't know that some people actually set their portals to something as strange as WINDOWS-1251, but that's why I asked about your experience. I couldn't honestly do the UTF-8 integration work without first looking at all the possible problems.
#16
Actualizado por Ivan Tcholakov hace alrededor de 1 año

  • Fichero 20081031_main_api_extensions.zip añadido

Comment

The plugin "Old Courses Restorer" is ready. I have decided to send to .
I hope it has been received there.

During its tests I doscovered that the function html_entity_decode() needs a replacement too.
So, there some changes were necessary to the extra-API I've sent here.

I am sending here the modified extra-API. List of changes:

main_api_encodings.lib.php - Gramatc correction within a comment (nothing iportant is here).
main_api_get_lang.lib.php - api_html_entity_decode() is used in one place.
main_api_string_conversion.lib.php - Implementation of api_html_entity_decode();
some additions to the comments have been added.

Well, this is all I have got on this topic.

With best regards,
Ivan Tcholakov
#17
Actualizado por Ivan Tcholakov hace alrededor de 1 año

Comment

If you test the plugin under XAMPP, most probably you will see this sudden strike:

Fatal error: Cannot redeclare class HTML_Table in C:\Users\UserFirstName UserLastName\workspace\dokeos\main\inc\lib\pear\HTML\Table.php on line 65
(or something similar)

I discovered this some hours ago.

Follow the advice I have given to Alessandro here: http://www.dokeos.com/forum/viewtopic.php?p=47041
and the plugin will work.

Two visible problems with the same reason.

Ivan Tcholakov
#18
Actualizado por Marko Kastelic hace alrededor de 1 año

Comment

Hi Ivan,
looks like extensions administration is a bit slow; will you be so nice and send the Old Courses Restorer plugin to mark111[AT]email[dot]si
MArk

btw.:like your main api extensions. So far everything works like charm.
#19
Actualizado por Ivan Tcholakov hace alrededor de 1 año

Comment

Hello, Marko!

I sent the original file to you. In a following message I sent to you and to
a slightly upgraded version as it is now in my place. Please, use the second one.

The upgrade is targeted to be avoided a problem that with different symptoms is described here:
http://www.dokeos.com/forum/viewtopic.php?p=47041

With best regards,
Ivan
#20
Actualizado por Yannick Warnier hace alrededor de 1 año

Comment

Regarding the adoption of extensions, it seems that we have a bottleneck whereby get to Patrick Cool and Patrick cannot get the extensions easily on the website because we removed the possibility to use PhpMyAdmin on that server (for security reasons) and because the command line SQL is a little too tedious to actually add the extensions. This being said, as soon as I have some time, I'll try to look into this as I know there are at least 3 extensions waiting for integration now (and a few comments to be added, like on the gradebook extensions which is now integrated into Dokeos 1.8.5 and 1.8.6).
#21
Actualizado por Yannick Warnier hace alrededor de 1 año

Comment

Ivan, can you send the extension to me directly? I'm right in the middle of putting a bunch of extensions on the website, so you might as well take benefit of that (annual) opportunity :-)
#22
Actualizado por Ivan Tcholakov hace alrededor de 1 año

Comment

Sent at the address I got from your profile here.
The second version (alpha2) is the actual one in my place.
#23
Actualizado por Yannick Warnier hace alrededor de 1 año

Comment

http://www.dokeos.com/extensions/index.php?section=tools
#24
Actualizado por Ivan Tcholakov hace 10 meses

Comment

Related tasks: FS#3442, FS#3446.
#25
Actualizado por Ivan Tcholakov hace 7 meses

Comment

A personal opinion:
Looks like UTF-8 support gets better. But finishing this task will be possible for Dokeos 1.8.7. This task has to be re-scheduled.
#26
Actualizado por Yannick Warnier hace 6 meses

Comment

Agreed. Internally, we have a few scripts to change the language files to UTF-8, which makes it easier to test. We have also setup http://i18n.dokeos.com to be an image of the dokeos trunk (refresh every 4 hours). Try it out there and report here :-)
#27
Actualizado por Ivan Tcholakov hace 6 meses

Comment

I am preparing the library to be in a single file. Looks like some 1.8.6 tasks will need it.
#28
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20419 - Introducing a new library for dealing with multibyte strings. The library has been implemeted as an extension of the main API.
#29
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20431 - The multibyte string library: Minor changes.
#30
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20432 (FS#306 and FS#3958) - Improvements in encoding management for the Social network tool.
#31
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20437 - The multibyte string library: Corrections for suppressing useless warning messages.
#32
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20438 (FS#306 and FS#3969) - Oogie converter: Some fixes for better encoding management.
#33
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20439 (FS#306 and FS#3958) - The social network tool: Fixing an encoding problem, group_contact.inc.php, line 27.
#34
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20440 (FS#306 and FS#4143) - Course (Learning path) tool: Some fixes for better character encoding management.
#35
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20441 - Platform administrator's tool: Improvements for better encoding management (UTF-8 too).
These are trivial changes that have been tested by me for a long time. Other will follow.
#36
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20442 - The announcements tool: Improvements for better encoding management.
#37
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20443 - The authentication modules: Improvements for better encoding management.
#38
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20444 - The Agenda tool: Improvements for better encoding management.
#39
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20445 - The Chat tool: Improvements for better encoding management.
#40
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20446 - The Conference tool: Improvements for better encoding management.
Looks like the file get_translation.lib.php is old and needs some reworks, but I did not touch it.
#41
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20447 - The Course description tool: Improvements for better encoding management.
#42
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20448 - The Course restorer: An improvement for better encoding management.
#43
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20449 - The Training (course) creator: An improvement for better encoding management.
#44
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20450 - The Dropbox tool: Improvements for better encoding management.
#45
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20451 - The Tests tool: Improvements for better encoding management.
#46
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20452 - The "Forums" tool: Improvements for better encoding management.
#47
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20453 - The Glossary tool: A modification for better encoding support.
#48
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20454 - The Assessments (Gradebook) tool: Improvements for better encoding support.
#49
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20455 - The "Groups" tool: Improvements for better encoding support.
#50
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20456 - The common libraries: Improvements for better encoding support, part 1.
#51
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20457 - The common libraries: Improvements for better encoding support, part 2.
#52
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20458 - Installation scripts: Shallow improvements for better encoding support.
#53
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20459 - The "Links" tool: Improvements for better encoding support.
#54
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20460 - The "Messages" tool: Improvements for better encoding support.
#55
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20461 - The metadata module: Improvements for better encoding support.
#56
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20463 - The reporting module: Improvements for better encoding support.
#57
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20465 - The "Courses" tool again: Improvements for better encoding support.
#58
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20467 - The html-implemented conferention tool: Improvements for better encoding support.
#59
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20468 - The resource linker: Improvements for better encoding support.
#60
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20470 - The "Surveys" tool: Improvements for better encoding support.
#61
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20472 - The tracking tool (obsolete?): Improvements for better encoding support.
#62
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20473 - The "Users" tool: Improvements for better encoding support.
#63
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20474 - The "Wiki" tool: Improvements for better encoding support.
#64
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20475 - The "Assignments" tool: Improvements for better encoding support.
#65
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20476 - Top level files check: Improvements for better encoding support.

This wave of commits is over. After a day I will start to test these changes. Feel free to complain here, guys.
#66
Actualizado por Ivan Tcholakov hace 6 meses

Comment

FS#4185 - Step position in learnig path
An accent problem has been reported there.
#67
Actualizado por Marko Kastelic hace 6 meses

Comment

The social network tool is not done yet (only group contacts are improved for now). Look the [http://projects.dokeos.com/index.php?do=details&task_id=4187&project=1&order=lastedit&sort=desc]
#68
Actualizado por Ivan Tcholakov hace 6 meses

Comment

Looks like the function get_lang() needs upgrade. I think, the encoding of the translation results does not fit.
Obviously, it is not needed both UTF-8 and non-UTF-8 versions of the language files to be supported simultaneously. One set is enough.
For Bulgarian language (non-UTF-8 files) I do conversion of get_lang() results "on the fly":

language variable id -----> get_lang() -----> chekck UTF-8 validance ------> recode the string if it is needed ------> decode htmlentities -----> translated result

Thus I use a single set of the language files for WINDOWS-1251 or UTF-8 sites. This trick may be done for the mainstream code. I want however to avoid checking every string for UTF-8 validance. I need an "official" and supported in the future way for detection what kind are translations for a given language - UTF-8 or not?

If the suffix _unicode disappears, what clue I to use? Anyway, for this release (1.8.6) I will upgrade get_lang() somehow.

I imagine the following final scenario for the translation system. All the translation files are UTF-8 encoded, without htmlentities inside. When the site is UTF-8 translated strings are returned directly. When the site is ISO-8859-1, ISO-8859-15, WINDOWS-1250, WINDOWS-1251, ..., the returned translated strings are to be converted.
#69
Actualizado por Ivan Tcholakov hace 6 meses

Comment

About the translation trick, why it looks complex: In the past Bulgarian language files were a mixture of strings encoded as WINDOWS-1251, UTF-8 and htmlentities.
#70
Actualizado por Ivan Tcholakov hace 6 meses

Comment

In a limited number of places there is still the following problem - some of the language variables are used directly without coverage by get_lang(). I will scan the code for such cases.
#71
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20539 - Upgrading the function get_lang() to return translated strings using the system encoding, nevertheless in what version (unicode or non-unicode) the language files are. Also, html-entities are converted into normal characters.
#72
Actualizado por Ivan Tcholakov hace 6 meses

Comment

...Yes, the function get_lang() is smart and fast enough. It may be tested for performance in a page with a large table inside which there are icons with tooltips. For long series of repetitive translations caching helps, see FS#3260 and FS#4046.
#73
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20549 (FS#306 and FS#4188) - Fixing a reported encoding problem, the FCKEditor, "MP3" plugin.
#74
Actualizado por Ivan Tcholakov hace 6 meses

Comment

Yannick, I have a request about the English and the Spanish installation manuals. It is well some note like the following to be mentioned:

"The setting "Administration > Configuration settings > Character set" may be changed while a newly installed system is still empty, without loaded trainings (courses). This setting should not be changed anymore after initial choice of the character set and creaing the first training (course) within the system."

Theoretically speaking, there is no abstraction level for the database where we could intercept the queries and the returned results and to recode them. At least I don't see such thing and also I don't see a practical reason it to be created.

About testing encoding issues:

A tester has to work on one installation with non-UTF-8 encoding (ISO-8859-15 for example) and a separate, another one with UTF-8 encoding. When on both systems the test is OK, then the targeted encoding problem may be considered as solved.
#75
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20557 - Preparing for testing the social tool about encoding issues. First, rewritting the function api_is_xml_http_request() in a safe way.
#76
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20570 - Introducing a string conversion function that might be suitable for serving XtmlHttp-requests.
#77
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20579 (FS#306, FS#4070 and FS#4187) - A proposal for reworking some methods of the Display class.
#78
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20580 (FS#306, FS#4070 and FS#4187) - A correction of a previous proposal for reworking some methods of the Display class.
#79
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20587 (FS#306 and FS#4187) - Fixing character encoding issues in the "Messages" and "Social network" tools, part 1.
#80
Actualizado por Ivan Tcholakov hace 6 meses

Comment

Here I am publishing some kind of documentation of the library of the
newly introduced library containing enhanced string functions:
.../dokeos/main/inc/lib/multibyte_string_functions.lib.php

// File: multibyte_string_functions.lib.php
// Main API extension for Dokeos 1.8.6 LMS
// A common purpose library for supporting multibyte string aware functions.
// This library requires PHP mbstring extension to be activated.
// When encodings to be used are not supported by mbstring, this library
// is able to exploit the PHP iconv extesion, which in this case should
// activated too.

// Notes:
//
// 1. For these functions where $encoding is not mandatory parameter,
// the system's encoding is assumed, i.e. the value that is returned by
// api_get_setting('platform_charset') or the value of the global
// variable $charset.
//
// 2. In other aspets, most of the functions in this library try to copy
// behaviour of some core PHP functions and some functions from the
// mbstring extension. Mostly they have similar names prefixed with "api_".
// For your convenience links have been given to the documentation of the
// original PHP functions.
//
// 3. Why these function have been introduced? Because the are able to
// support more encodings than the original ones. And which is most
// important - they are UTF-8 aware. So, they should be used for strings
// in natural language. For internal system identificators of file names
// which are supposed to contain only English letters you may use the
// original PHP string functions.
//
// 4. This is not a full list, functions for internal or upgrade purposes
// are not mentioned here, they are not important from developer's
// perspective.

// ----------------------------------------------------------------------------
// A safe way to calculate binary lenght of a string (as number of bytes)
//----------------------------------------------------------------------------

// Calculates binary lenght of a string, as number of bytes, regardless the php-setting mbstring.func_overload.
function api_byte_count($string) ----------------------------- Added 06-JUN-2009, Dokeos 1.8.6.1 -------------------------

//----------------------------------------------------------------------------
// Multibyte string conversion functions
//----------------------------------------------------------------------------

// Converts character encoding of a given string.
// See http://php.net/manual/en/function.mb-convert-encoding
function api_convert_encoding($string, $to_encoding, $from_encoding)

// Converts a given string into UTF-8 encoded string.
// See http://php.net/manual/en/function.utf8-encode
function api_utf8_encode($string, $from_encoding = null)

// Converts a given string, from UTF-8 encoding to a specified encoding.
// See http://php.net/manual/en/function.utf8-decode
function api_utf8_decode($string, $to_encoding = null) {

// Encodes a given string into the system ecoding if this conversion has been detected as necessary.
function api_to_system_encoding($string, $from_encoding = null, $check_utf8_validity = false)

// Converts all applicable characters to HTML entities.
// See http://php.net/manual/en/function.htmlentities
function api_htmlentities($string, $quote_style = ENT_COMPAT, $encoding = 'ISO-8859-15')

// Decodes HTML entities into normal characters.
// See http://php.net/html_entity_decode
function api_html_entity_decode($string, $quote_style = ENT_COMPAT, $encoding = 'ISO-8859-15')

// This function encodes (conditionally) to UTF-8 a given string if XmlHttp-request has been detected.
function api_xml_http_response_encode($string, $from_encoding = null)

// This function converts a given string to the encoding that filesystem uses for representing file/folder names.
function api_file_system_encode($string, $from_encoding = null) ----------------------------- Added 03-JUN-2009, Dokeos 1.8.6.1 -------------------------

// This function converts a given string from the encoding that filesystem uses for representing file/folder names.
function api_file_system_decode($string, $to_encoding = null) ----------------------------- Added 03-JUN-2009, Dokeos 1.8.6.1 -------------------------

//----------------------------------------------------------------------------
// Common multibyte string functions
//----------------------------------------------------------------------------

// Regular expression match with multibyte support.
// See http://php.net/manual/en/function.mb-ereg
function api_ereg($pattern, $string, & $regs = null)

// Replace regular expression with multibyte support.
// See http://php.net/manual/en/function.mb-ereg-replace
function api_ereg_replace($pattern, $replacement, $string, $option = null)

// Regular expression match ignoring case with multibyte support.
// See http://php.net/manual/en/function.mb-eregi
function api_eregi($pattern, $string, & $regs = null)

// Replace regular expression with multibyte support ignoring case.
// See http://php.net/manual/en/function.mb-eregi-replace
function api_eregi_replace($pattern, $replacement, $string, $option = null)

// This function returns a selected by position character of a string.
function api_get_character($string, $position, $encoding = null) ------- REMOVED 2009-05-15 --------

// This function returns an array containing all characters of a string.
function api_get_characters($string, $encoding = null) ------- REMOVED 2009-05-15 --------

// Makes a string's first character lowercase.
// See http://php.net/manual/en/function.lcfirst
function api_lcfirst($string, $encoding = null) ------- REMOVED 2009-05-16 --------

// Splits string into array by regular expression.
// See http://php.net/manual/en/function.mb-split
function api_split($pattern, $string, $limit = null) ------- REMOVED 2009-05-15 --------

// This is a multibyte replacement of str_ireplace().
// See http://php.net/manual/en/function.str-ireplace
function api_str_ireplace($search, $replace, $subject, & $count = null, $encoding = null)

// This is a multibyte replacement of str_split().
// See http://php.net/str_split
function api_str_split($string, $split_length = 1, $encoding = null)

// This is a multibyte replacement of stripos().
// See http://php.net/manual/en/function.mb-stripos
function api_stripos($haystack, $needle, $offset = 0, $encoding = null)

// This is a multibyte replacement of stristr().
// See http://php.net/manual/en/function.mb-stristr
function api_stristr($haystack, $needle, $part = false, $encoding = null)

// Returns length of the input string.
// See http://php.net/manual/en/function.mb-strlen
function api_strlen($string, $encoding = null)

// This is a multibyte replacement of strpos().
// See http://php.net/manual/en/function.mb-strpos
function api_strpos($haystack, $needle, $offset = 0, $encoding = null)

// This is a multibyte replacement of strrchr().
// See http://php.net/manual/en/function.mb-strrchr
function api_strrchr($haystack, $needle, $part = false, $encoding = null)

// This is a multibyte replacement of strrev().
// See http://php.net/manual/en/function.strrev
function api_strrev($string, $encoding = null)

// This is a multibyte replacement of strrpos().
// See http://php.net/manual/en/function.mb-strrpos
function api_strrpos($haystack, $needle, $offset = 0, $encoding = null)

// This is a multibyte replacement of strstr().
// See http://php.net/manual/en/function.mb-strstr
function api_strstr($haystack, $needle, $part = false, $encoding = null)

// Makes a string lowercase.
// See http://php.net/manual/en/function.mb-strtolower
function api_strtolower($string, $encoding = null)

// Makes a string uppercase.
// See http://php.net/manual/en/function.mb-strtoupper
function api_strtoupper($string, $encoding = null)

// Translates certain characters.
// See http://php.net/manual/en/function.strtr
// It would be good tihs function to be removed. I hesitate to do it right now.
function api_strtr($string, $from, $to = null, $encoding = null)

// Performs a multi-byte safe substr() operation based on number of characters.
// See http://bg.php.net/manual/en/function.mb-substr
function api_substr($string, $start, $length = null, $encoding = null)

// Puts a suffix into a string.
// The input variables could be arrays too.
function api_suffix($string, $suffix) ------- REMOVED 2009-05-15 --------

// This is a multibyte replacement of substr_replace().
// See http://php.net/manual/function.substr-replace
function api_substr_replace($string, $replacement, $start, $length = null, $encoding = null)

// Returns a string with the first character capitalized, if that character is alphabetic.
// See http://php.net/manual/en/function.ucfirst
function api_ucfirst($string, $encoding = null)

// Uppercases the first character of each word in a string.
// See http://php.net/manual/en/function.ucwords
function api_ucwords($string, $encoding = null)

// This function adds a unicode modifier to a
// Perl-compatible regular expression when it is necessary.
function api_add_pcre_unicode_modifier($pcre, $encoding = null)

/** * ---------------------------------------------------------------------------- * Functions for string comparison * ----------------------------------------------------------------------------
*/

// This is a multibyte replacement of strcasecmp().
// See http://php.net/manual/en/function.strcasecmp
function api_strcasecmp($string1, $string2, $language = null, $encoding = null) ----------------- Parameters changed 18-JUN-2009, Dokeos 1.8.6.1 ----------------

// This is a multibyte replacement of strcmp().
function api_strcmp($string1, $string2, $language = null, $encoding = null) ----------------------------- Added 17-JUN-2009, Dokeos 1.8.6.1 -------------------------

//----------------------------------------------------------------------------
// Encoding management functions
//----------------------------------------------------------------------------

// Returns the most-probably used non-UTF-8 encoding for the given language.
// The $language parameter must be cleaned by api_refine_language_id() if it
// is necessary.
// If the returned value is not as you expect, you may do the following:
// In the table $encodings below, correct the order of the encodings for your
// language, or if it is necessary - insert at the first place a new encoding.
function api_get_non_utf8_encoding($language = null)

// This function checks whether two $encoding are equal (same, equvalent).
function api_equal_encodings($encoding_1, $encoding_2)

// Returns true if the given encoding id means UTF-8, otherwise returns false.
function api_is_utf8($encoding)

// This function checks whether a given encoding represents (is an alias of) ISO Latin 1 character set.
function api_is_latin1($encoding) ----------------------------- Added 12-JUN-2009, Dokeos 1.8.6.1 -------------------------

// Returns the encoding currently used by the system.
function api_get_system_encoding()

// This function returns the encoding, currently used by the file system.
function api_get_file_system_encoding() ----------------------------- Added 03-JUN-2009, Dokeos 1.8.6.1 -------------------------

// Sets all internal default encodings of the multi-byte functions to the given value.
function api_initialize_internal_encoding($encoding) ----------------------------- Added 15-JUN-2009, Dokeos 1.8.6.1 -------------------------

//----------------------------------------------------------------------------
// String validation functions concerning some encodings
//----------------------------------------------------------------------------

// Returns true if the specified string is a valid UTF-8 one and false otherwise.
function api_is_valid_utf8($string)

// Checks whether a string contains 7bit ASCII characters only.
function api_is_valid_ascii($string)

//----------------------------------------------------------------------------
// Language management functions
//----------------------------------------------------------------------------

// Returns a pure language id, without possible suffixes
// that will disturb language identification in certain cases.
function api_refine_language_id($language)

// This function check whether a given language can use Latin 1 encoding (ISO-8859-15, ISO-8859-1, WINDOWS-1252, ...).
function api_is_latin1_compatible($language) ----------------------------- Added 15-JUN-2009, Dokeos 1.8.6.1 -------------------------

// This function returns an array of those languages that can use Latin 1 encoding (ISO-8859-15, ISO-8859-1, WINDOWS-1252, ...).
function api_get_latin1_compatible_languages() ----------------------------- Added 15-JUN-2009, Dokeos 1.8.6.1 -------------------------

//----------------------------------------------------------------------------
// Array functions
//----------------------------------------------------------------------------

// A case insensitive version of in_array() function.
// See http://php.net/manual/en/function.in-array.php
function api_in_array_nocase($needle, $haystack, $strict = false, $encoding = null)

//----------------------------------------------------------------------------
// Transliteration, converting ANSI and UTF-8 strings to ASCII strings
//----------------------------------------------------------------------------

// Transliterates a string to a plain ASCII string.
function api_transliterate($string, $unknown = '?', $from_encoding = null) ----------------------------- Added 03-JUN-2009, Dokeos 1.8.6.1 -------------------------
#81
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20624 (FS#306 and FS#4187) - The multibyte string library: Adding some notes in the header. Modification of the functions api_htmlentities() and api_html_entity_decode() to fit better with current development habits. Simplifying the functions api_utf8_encode() and api_utf8_encode().
#82
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20629 - The multibyte string library: A correction of the previous commit SVN#20624.
#83
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20631 - The multibyte string library: Reverting back to the previous implementations of api_utf8_encode() and api_utf8_decode() - they work faster. Starting to comment the library in PHPDoc style.
#84
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20665 (FS#306) - The multibyte string library: Now the library is documented in PHPDoc style. See the file .../dokeos/main/inc/lib/multibyte_string_functions.php.
#85
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20677 - The multibyte string library: Adding a comment about the function api_strlen().
#86
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20682 - Replacing some natural language unsafe occurrences of the functions strlen() and substr(). Fixing a call to a non-existing function.
#87
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20693 - Replacing some natural language unsafe occurrences of the functions strtolower() and strtoupper().
#88
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20696 - A correction about Russian language: The prefered non-UTF-8 encoding has been changed to be KOI8-R. The translations are visible correctly using this encoding.
#89
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20699 - Session import: Replacing of some strpos() function to api_strpos() nad some other functions around. Probably this is needed. If not, replacements do not change behaviour.
#90
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20709 - Replacing some natural language unsafe occurrences of the function ucfirst().
#91
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20716 - Replacing some natural language unsafe occurrences of the function htmlentities() and other similar fixes.
#92
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20717 - Replacing some natural language unsafe occurrences of the function html_entity_decode().
#93
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20720 - Replacing some natural language unsafe occurrences of the function utf8_encode().
#94
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20723 - Replacing some occurrences of the function mb_convert_encoding() with api_convert_encoding() which is able to support more encodings if the iconv php-extension has been activated.
#95
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20724 -The multi-byte string library: A minor correction within the function api_get_non_utf8_encoding().
#96
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20725 - The Conferecnce tool, a proposal for a change within the function get_language_file_as_xml().
#97
Actualizado por Ivan Tcholakov hace 6 meses

Comment

I have a request. Is it possible all the language files to be placed within the system in http://i18n.dokeos.com ? It doesn't matter whether they are unicode or not, they all should work.

At the moment I can switch into Chinese for example, but I can not switch into many other languages.
#98
Actualizado por Ivan Tcholakov hace 6 meses

Comment

And about http://dev.dokeos.com : There is no chance Chinese to be seen using iso-8859-15 encoding. On this site only languages of the Latin1 group could be used. I am mentioning this just in case, if somebody asks why... :-)
#99
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20728 - The multi-byte string library: Optimizing the function api_refine_language_id().
#100
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20729 - The multi-byte string library: Removing the function api_lcfirst(), it is useless.
#101
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20747 - The Messaging tool and the Display class: Fixing some encoding management problems.
#102
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20748 - Fixing direct use of the language variable $langQuestion. Now its value is processed and retrieved by the function get_lang().
#103
Actualizado por Ivan Tcholakov hace 6 meses

Comment

About Hotspot tests: I thied to enable languages with non-Latin origin without success. I opened a separate task FS#4222.
#104
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20753 - The multi-byte sting library: Updating the internal table of language-specific encodings.
#105
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20756 (FS#306 and FS#4224) - Fixing an encoding problem for the "Documents" tool, metadata form. For non-UTF-8 platforms this fix will work with ISO-8859-1, ISO-8859-15, and WINDOWS-1252 encodings, i.e. Latin1 language group will be supported only. For UTF-8 platforms all the languages will be supported. The restriction is due to the xml-parser implementation, see http://php.net/manual/en/function.xml-parser-create-ns.php .
#106
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20799 - "Freezing" the multi-byte string library for the incoming release Dokeos 1.8.6.
#107
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20818 (FS#306 and FS#4226 ) - SCORM export: Removing some tabulator and new-line control characters that are converted into html-entities within imsmainfest.xml file.
#108
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20915 - The Learning path tool: Eliminating some conversions with identical "from" and "to" encodings.
#109
Actualizado por Ivan Tcholakov hace 6 meses

Comment

Although it is not quite relevant:
SVN#20916 - The Wiki tool: Inserting a space between first name and last name.
Order of a person's names has to be managed, related task: FS#3646.
#110
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20923 - The multi-byte string libray: Removing references to some rare encodings that are not supported by the iconv library.
I intend not to make further activities in task FS#306 until the 1.8.6 release. It is about a week left for testing.
#111
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#20950 - The Chat tool: Declaring the system encoding for the main page that contains the frames. Thus, damaged text will not appear in the browser's title-bar.
#112
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#21055 - A fix: Moving code that includes the multi-byte string library after the section that declares constants. For PHP 5.2.9-2 (tested on Windows Vista, IIS7, FastCGI) notices were produced. The constant LIBRARY_PATH has to be declared first.
#113
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#21070 - Dropbox has its own implementation for language translation. Making the function dropbox_lang() compatible with the new logic for using language files: The correct output encoding should be guessed and htmlentities should be converted into normal characters.
#114
Actualizado por Marko Kastelic hace 6 meses

Comment

ok, 1.8.6 is out. Nice progress for utf-8 support.
There's a kind of mess in documents tool since 1.6.2 (i believe). Let me invite you to look to the documents repository at the i18n server (course = 00TEST00). Do not look at the file names, just try to access to some of the uploaded in some locally created (magic wand at the right) files.

Some observations on files with filenames that includes non-latin1 chars:

a) when (i) upload the file with .html extension, new .htm suffix is added to the file name ?
b) if i create new html file on the platform whose name includes non-latin1 characters, i cannot access to the file (open or download) but can access to the same file (edit)
c) some characters (eq. s with carrot) is automatically converted to s (without carrot) ?

d) when opening non-html file (eq. txt) and file cannot be accessed (note b ) platform silently dies with no warning (can't find the file)

again, this tool is not usable out-of-the-box. For now, i'll try to "fix" filenames to something we call yuscii charset originated from 1970's (fit within ascii7) to make platform usable.

and another : please, do not 'fix' dropbox to much. At the moment uploads/download works like charm. With utf-8 enabled, filenames are strange, but who looks at them anyway ?
#115
Actualizado por Ivan Tcholakov hace 6 meses

Comment

Hello Marko,

For the 1.8.6 release I did not implement encoding mechanism for file system names, the situation with them is as it was for now. I preferred go start with a relatively small step to cover most urgent issues. This task has 20% progress at the moment, it will be continued, of course.

Concerning managing encodings for file system names the following topics I have to think about:
1. Performance - it is not very clear to me whether performance will suffer. This has to be checked.
2. Portability - this is the most difficult issue. So called "locales" are different for Windows and Linux, they are not well standartized.
3. PHP6 promises to solve this problems, the ICU library will be natively supported. There is a probability we to waste our strength for reinventing the wheel.
4. Anyway, maybe it will be a good idea we to create a Filesystem class similar to those in Dokeos 2.0 alpha. Inside it we may hide managing encodings and dealing with file/folder permissions (that are set for the platform as default).
5. Another way (I don't like it) is to use transliteration for file system names.

It is time for the "stop & think" exercise. :-) A week will be enough.
#116
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#21217 - Introducing a new function api_get_file_system_encoding(). Guessing file system encoding is tricky, some corrections in this function are possible.
#117
Actualizado por Ivan Tcholakov hace 6 meses

Comment

Some interesting links about UTF-8 and internationalization:

Ivan Tcolakov, 16-SEP-2009: Removed for avoiding confusion, see the updated list below.
#118
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#21246 - Introducing two conversion functions: api_file_system_encode() and api_file_system_decode().
#119
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#21249 - Introducing a new library that will re-implement some PHP file system related functions. These replacement functions will deal with the possibly diffenrent encodings that are used by the Dokeos platform and the server's file system.

The idea became clear:

1. We use some PHP functions that are related to the file system. Let these primitive functions to be re-implemented, so they can deal with encodings properly. The inserted code demonstrates this approach.
2. (Optional) A Filesystem class may be implemented, its methods could be on higher semantic level that is close to the needs of the Dokeos system. I like the Filesystem class in the Dokeos 2.0 alpha.
#120
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#21254 - Adding the functions api_is_dir() and api_is_system_path() within the new low-level API to the file system. A minor fix for the function api_url_to_local_path() within the main API.
#121
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#21272 - Introducing more functions in the file system API.
#122
Actualizado por Ivan Tcholakov hace 6 meses

Comment

SVN#21273 - A minor addition as comment - a reference link has been added inside the file system API.
#123
Actualizado por Ivan Tcholakov hace 6 meses

Comment

Some important forum topics:

http://www.dokeos.com/forum/viewtopic.php?t=2759
http://www.dokeos.com/forum/viewtopic.php?t=3119
#124
Actualizado por Ivan Tcholakov hace 5 meses

Comment

SVN#21274 - The multi-byte string library: Adding a function that calculates correctly length of binary data (as number of bytes) regardless the PHP configuration (See the ini-setting mbstring.func_overload). This function may be upgraded for PHP6 in the future.
#125
Actualizado por Ivan Tcholakov hace 5 meses

Comment

Dear Marko,

I would like to express my opinion about enabling non-english characters in file names. It is impossible work for Dokeos 1.8.6.1. Too much code has to be changed and tested. It is difficult to me at this moment to alter some of the previous hacks. Also, I have to think now about the upgrade scripts in the future versions, the scripts for backup/restore courses, SCORM export, etc. Altering the current way for processing filenames will create burden for the developers of Dokeos 2.0, they would have to adjust their upgrade scripts too. There are fragments of commersial code inside the system that I am not able to test after my changes (for example, the indexer for the search machine).

I prefer for Dokeos 1.8.6.1 to be done only bugfixes on the UTF-8 functionality we already have. As I estimate, my personal efforts are not enough to change the situation here for the next release.

Too much work for just having non-English file names looks to me wasting time, it is not a good deal. Rumor says PHP6 will have stable release this year, and with its native UTF-8 support our hacks will be not necessary.

Porting Dokeos 1.8.x to PHP6 (with solving the filenames problem and other internationalization issues) looks to me more perspective effort.
#126
Actualizado por Ivan Tcholakov hace 5 meses

Comment

There is another problem to be researched - whether it is possible or not the PclZip library to support UTF-8 encoded file names.
#127
Actualizado por Marko Kastelic hace 5 meses

Comment

ok, thanks to let me know. i'm moving on translationg file names to english-like filenames (without accented chars).
#128
Actualizado por Ivan Tcholakov hace 5 meses

Comment

SVN#21360 - Cancellation of the new file system API, cleaning the file system library. There is no rational reason to go this way, too much work is needed.
#129
Actualizado por Ivan Tcholakov hace 5 meses

Comment

SVN#21361 - Various fixes for better character encoding support.
#130
Actualizado por Ivan Tcholakov hace 5 meses

Comment

SVN#21362 - Fixing direct output of a language variable, the function get_lang() should be used.
#131
Actualizado por Ivan Tcholakov hace 5 meses

Comment

SVN#21382 - The multi-byte string library: Making the function api_is_valid_utf8() unaffected by the PHP ini-setting mbstring.func_overload.
#132
Actualizado por Ivan Tcholakov hace 5 meses

Comment

SVN#21389 - Adding a boolen parameter $purified to the function api_get_interface_language(). For example, when $purified is true, the function will return 'french' instead of 'french_unicode'. In certain cases we need such purified or refined language identificators.
#133
Actualizado por Ivan Tcholakov hace 5 meses

Comment

SVN#21392 - The multi-byte string library: Adding a function that performs transliteration from UTF-8 to ASCII. Adding also a required database with character replacements. The solution has been adapted from the source code of the Drupal's module "Transliteration", version 6.x-2.1, 09-JUN-2009.
#134
Actualizado por Ivan Tcholakov hace 5 meses

Comment

SVN#21394 - The multi-byte string library: Corrections in the comments about the transliteration function. Renaming a variable according to our rules.
#135
Actualizado por Ivan Tcholakov hace 5 meses

Comment

SVN#21405 - The multi-byte string library: Adding the function api_is_latin1($encoding) for detection aliases of the Latin 1 character set - ISO-8859-15, ISO-8859-1, WINDOWS-1252, CP1252, ... . Typically this function will be needed for code that provides backward compatibility for Latin 1 Dokeos sites.
#136
Actualizado por Ivan Tcholakov hace 5 meses

Comment

SVN#21406 - Modification of the function replace_dangerous_char(). For Latin 1 platforms (ISO-8859-15, ISO-8859-1, WINDOWS-1252, ...) its behaviour has been preserved. For other platforms (including UTF-8) safe ASCII file names are generated through using transliteration.
#137
Actualizado por Ivan Tcholakov hace 5 meses

  • Fichero cyrillic-name-file-system.png añadido * Fichero the-same-name-transliterated-in-dokeos.png añadido

Comment

Transliteration got functional, see a test on the attaced pictures. Platform encoding UTF-8, file system encoding UTF-8, OS Ubuntu 9.04.
#138
Actualizado por Ivan Tcholakov hace 5 meses

Comment

A pure transliteration example:

? ?????? ?? ---> Fyodor Mihaylovich Dostoevkiy

Transliteration may be used not only for file names. It may be applied for proposing course codes, usernames, etc.

Update 13-JUN-2009:

Another experiment on the same system: echo api_transliterate('????', 'UTF-8'); ---> Xiao Yuan Shou Ye

Update 14-SEP-2009: :-) Something has gone wrong with the examples.
#139
Actualizado por Ivan Tcholakov hace 5 meses

Comment

SVN#21411 - The global initialization script: Commenting a duplicate code for initialization of the global variable $charset, to be removed. Offtopic: A minor correction within the table that serves the function add_ext_on_mime() - for 'text/html' MIME type the '.html' extension will be assigned instead of '.htm'.
#140
Actualizado por Ivan Tcholakov hace 5 meses

Comment

SVN#21412 - The file uploader: Fixing the function add_ext_on_mime() properly to dete

#2

Updated by Ivan Tcholakov about 10 years ago

Hello, guys. I am checking the account here.

#3

Updated by Ivan Tcholakov almost 10 years ago

I am going to add some code for finishing conceptually the internationalization library. Language and encoding detection will be added, also corresponding tests (disabled). It is well these features to become visible to the developers now.
This code does not change the behaviour of the system at the moment, don't worry.

#4

Updated by Ivan Tcholakov almost 10 years ago

9831:930333d3d2fe tip Ivan Tcholakov 2010-01-15 07:14 +0200 Feature #272 - The intenationalization library: Adding code for language and encoding detection support.

#5

Updated by Ivan Tcholakov almost 10 years ago

9914:9bec3d0efd93 Feature #272 - Adding code in the Database class for improved internationalization. The new code is latent, it will be used after the first release of Chamilo.
9913:960b34f655d9 Feature #272 - The intenationalization database: Updating/expanding the tables about non-UTF-8 encodings and about person name conventions. More languages are supported.

#6

Updated by Ivan Tcholakov almost 10 years ago

9915:ff66ecf05ddd Feature #272 - More code in the Database class for improved internationalization in the future.

#7

Updated by Ivan Tcholakov almost 10 years ago

We need a solution about the database encoding. After a short experiment I am proposing the following:

1. All the tables should be UTF-8 encoded.
2. The GUI setting "Platform Character Set" should stay. When it is not UTF-8, then the database server silently recodes the queries and the returned data.

Here is an illustration on how this recoding server feature could be exploited:

// Inserted experimental fragment in the installation script
mysql_query("SET SESSION character_set_server='utf8';");
mysql_query("SET SESSION collation_server='utf8_general_ci';");
mysql_query("SET CHARACTER SET '" . Database::to_db_encoding($charset) . "';");

// Inserted experimental fragment in the global initialization script
Database::query("SET SESSION character_set_server='utf8';");
Database::query("SET SESSION collation_server='utf8_general_ci';");
Database::query("SET CHARACTER SET '" . Database::to_db_encoding($charset) . "';");

See http://dev.mysql.com/doc/refman/5.0/en/charset-connection.html

After inserting these lines (+ some other changes) and installing the system I
created a Bulgarian course, UTF-8. Then I changed the platform character set to
WINDOWS-1251, as a result the Bulgarian texts are normal, not broken.

For example, a Spanish installation could be switched from UTF-8 to ISO-8859-15
without problems, if some Flash objects in the tests still not work with UTF-8.

Another advantage - making the database data to be always UTF-8 encoded would ease
the migration to Chamilo 2.0.

I like this approach. I am going to make stable code for this solution
and to implement it in the mainstream code. The upgrading script should support
this change.

#8

Updated by Ivan Tcholakov almost 10 years ago

A simple scheme about the database encoding has been added. It is according to the idea above.

#9

Updated by Ivan Tcholakov almost 10 years ago

10017:f5b05ee15814 Feature #272 - Training creation: Some hard-coded (through SQL) table properties has been removed.
10021:35be3076652c Feature #272 - Preparations for change in the installation script: Removing the obsolete file "language_data.csv" and related to it code.
10022:d6881ad79696 Feature #272 - Preparations for change in the installation script: Removing the obsolete files "setting_current_data.csv" and "setting_option_data.csv" and related to them code.

#10

Updated by Ivan Tcholakov almost 10 years ago

10023:d3760c526c17 Feature #272 - Preparations for change in the installation script: Shrinking the code about PHP version check.
10024:f443191851a4 Feature #272 - Preparations for change in the installation script: A useless file has been removed.
10025:c38bc9a98756 Feature #272 - Preparations for change in the installation script: Some minor corrections have been made.
10026:a19a80213fac Feature #272 - An encoding related logucal correction within the database class.
10027:45499db69dac Feature #272 - Mode encoding related corrections within the database class.
10028:fce3a026ed97 Feature #272 - Updating some comments.
10029:a7ab31e58695 Feature #272 - System fresh installation: Making database encoding to be UTF-8. Also platform character set will be UTF-8 by default. The upgrade script will be fixed later. Before testing, reinstall your system, please. If you wish, you may set your system to be ISO-8859-15, for that it is enough you to change the setting "Platform character set".

#11

Updated by Ivan Tcholakov almost 10 years ago

:-) Still there is work to be done. The good news is that the original UTF-8 enabled system works with Bulgarian language. You may enable and try some other non-latin languages too.

#12

Updated by Yannick Warnier almost 10 years ago

  • Target version changed from 1.8.7 to 1.8.7 beta

I think it's a good idea to plan this for the beta, in particular in terms of migrations from previous versions, which will need to include the migration of all textual data from the database. The best way to do this would be to analyse every text, char, varchar (and corresponding variations) fields to UTF-8 from the current character encoding, but this will possibly take ages. Another way is to ask the user to make a dump of its database and convert it to UTF-8 with a given tool, but this is expecting too much of the typical user... I would go for the first solution, but that would imply changing the encoding/collation of each table as well and I'm not sure all chamilo database users can do that in the config of all of our users... In any case, the documentation and the services providers must be ready by then!

#13

Updated by Yannick Warnier almost 10 years ago

I'll try to make more language packs available in UTF-8 by the middle of March. See #551

#14

Updated by Yannick Warnier almost 10 years ago

  • SCRUM pts - complexity changed from ? to 100
#15

Updated by Ivan Tcholakov almost 10 years ago

Feature #272 - Platform admin section, the platform character set option: Restricting the lis of the proposed encodings to those that MySQL supports natively.

Dear Yannick,
It would be very well language files to be converted to UTF-8. This will make translators' life easier. Current language files should work too, because the encoding of every requested string is tested and adjusted dynamically (get_lang).

About the upgrade: At the moment I think, maybe the script could be as usual with one difference. Two db connections could be opened. The first connection would read old data as it was in the previous versions. The second connection would write data in the destination system tables as UTF-8. This might make the things easy, I will see.

The newly created tables are configured properly, only the upgrading script is to be adapted. It does not look to me difficult. For sophisticated cases, the function api_detect_encoding() might help.

The documentation will be upgraded.

An experiment: Make a Spanish course on the new UTF-8 system (with accent characters). Switch the site to ISO-8859-15. Then the same course should work without breaking the accents. Data in the database is always UTF-8 encoded, the server recodes data while it communicates.

#16

Updated by Ivan Tcholakov almost 10 years ago

10042:3659a7fce855 Feature #272 - Cosmetics in mail functions.
10040:bbc6ca4a75ee Feature #272 - A correction/reverting someone's change about e-mail notification by the platform admin.
10039:ae5dc09d5572 Feature #272 - PHPMailer, adding empty files for protection.
10038:a5b7d02ef16e Feature #272 - Modifications in PHPMailer class in order it not to depend on the mbstring library.
10037:dc52eeaa4e15 Feature #272 - Upgrading PHPMailer to version 5.1, 11-NOV-2009. (part 2)
10036:204d723489fc Feature #272 - Upgrading PHPMailer to version 5.1, 11-NOV-2009.
10035:94473ac130ab Feature #272 - Code-formatting, file mail.lib.inc.php.
10034:0ff011a2ddfe Feature #272 - Changing a mail setting to be more convenient.
10033:97edcf05e30d Feature #272 - Corrections in the function api_send_mail() (which is in the main API).
10032:9f1eee27ef40 Feature #272 - A correction in the confihuration file for the mailing sub-system.
10031:4c3e0a634be9 Feature #272 - Writting an author's name (Ren?) correctly within UTF-8 source code.
10030:bd53a36bf64e Feature #272 - Platform admin section, the platform character set option: Restricting the list of the proposed encodings to those that MySQL supports natively.

#17

Updated by Ivan Tcholakov almost 10 years ago

10049:3647df649ca0 Feature #272 - Mailing: The function api_send_mail() has been declared as deprecated.
10048:c6809de07bed Feature #272 - Various corrections about mailing functionality.
10047:b1e0e181b762 Feature #272 - Mailing: Corrections for the announcement tool. Still there are bugs, they should be fixed later.
10046:4127110eb42d Feature #272 - Mailing: Duplicate statements for loading the configuration file have been removed.
10045:57cd4da0ac00 Feature #272 - Mailing: Correction for loading the right configuration file. A global variable has been renamed for less probable conflicts.

#18

Updated by Ivan Tcholakov almost 10 years ago

10064:18ca2f6278b0 Feature #272 - Importing tcpdf library, version 4.8.027.
10052:ef765dbc86c6 Feature #272 - A correction in tracking/csv-export file (obsolete?).
10051:b3c4dd4dc60e Feature #272 - Corrections in some (obsolete?) files.
10050:10c76a7d8e25 Feature #272 - Various encoding-related corrections.

#19

Updated by Ivan Tcholakov almost 10 years ago

I need to solve the problem with PDF-generation in different languages (UTF-8, LTR-RTL).

The currently existing in the code libraries like ezpdf and html2pdf (with fpdf inside) are not solutions, because they do not support UTF-8. Period.

This is what I consider at the moment.

1. tcpdf , http://www.tecnick.com/public/code/cp_dpage.php?aiocp_dp=tcpdf ; documentation - http://www.tecnick.com/pagefiles/tcpdf/doc/com-tecnick-tcpdf/TCPDF.html
- it really supports UTF-8
- it is able to make pdf from html, ...
- ... but there is no CSS support. After a test I realized that this kind of export does not support CSS that comes with the page.

2. mPDF (based on FPDF, HTML2FPDF, and UFPDF, with a number of enhancements), http://mpdf.bpm1.com/ ; documentation - http://mpdf1.com/manual/index.php
- looks promising, according to the description it supports UTF-8 and CSS. I will test this library tomorrow.

#20

Updated by Ivan Tcholakov almost 10 years ago

Articles
-------------------

tcpdf

Creating PDF Documents in PHP Using Tcpdf - http://www.onemoretake.com/2009/02/28/creating-pdf-documents-in-php-using-tcpdf/
How to Create a Pdf in Php Using Tcpdf - http://www.coderdan.com/2009/12/how-to-create-a-pdf-in-php-using-tcpdf/
Generate PDF documents using TCPDF in Symfony - http://symfonynerds.com/blog/?p=158
Creating PDF files with CakePHP and TCPDF - http://bakery.cakephp.org/articles/view/creating-pdf-files-with-cakephp-and-tcpdf

ufpdf

UFPDF: Unicode/UTF-8 extension for FPDF - http://acko.net/node/56

Miscelaneous

Creating PDF Documents with PHP - http://talks.php.net/show/osconpdf/
Teaching an old PHP application to talk Spanish, Chinese & Hebrew - http://yetanotherprogrammingblog.com/node/15
Really Useful Classes And Libraries For PHP Developers - http://www.w3avenue.com/2009/08/11/really-useful-classes-and-libraries-for-php-developers/

#21

Updated by Ivan Tcholakov almost 10 years ago

10082:8965318dcbd8 Feature #272 - PDF export: TCPDF library (http://www.tecnick.com/public/code/cp_dpage.php?aiocp_dp=tcpdf), which was introduced for testing purposes, has been removed. I would not adopt it. At least I hesitate to do that. The next library for testing is mPDF (http://mpdf.bpm1.com).

#23

Updated by Ivan Tcholakov almost 10 years ago

I think mPDF is very good work, I started to like it. It is some kind of UTF-8-ready successor of FPDF+HTML2PDF. The API is almost the same. In the attached image a sample export in Bulgarian language is shown.

#24

Updated by Juan Carlos Raña Trabado almost 10 years ago

Ivan Tcholakov wrote:

I think mPDF is very good work, I started to like it. It is some kind of UTF-8-ready successor of FPDF+HTML2PDF. The API is almost the same. In the attached image a sample export in Bulgarian language is shown.

I know the package and look good, but I have not worked much with him. Would have to prove it. How supports UTF8? What minimum size should be up to Chamil to work well? ...
On the other hand, why hast thou cast TCPDF library?, what problem have you seen her?

#25

Updated by Ivan Tcholakov almost 10 years ago

Both TCPDF and mPDF support UTF-8. OK.

1. Both of the libraries are able to make pdf by using specialized API-commands,
of course.

2. Both of these libraries are able to produce pdf output from html code. Ok, but...
TCPDP can not interpret CSS styles. mPDF can deal with CSS, the layout of the
output pdf is very close to the original web page.
TCPDF gave me ugly output from html, this is why I rejected it.

3. mPDF is very easy to be installed. It is very easy to be used by a programmer.
If you know fpdf and html2pdf API's, this is an advantage, because mPDF uses
almost the same API.

I am preparing here the library (some files are to be deleted - examples, etc.).
I reworked the export from the Wiki tool. Tomorrow I will have a look to adapt the
export from the Gradebook. After that I will import the new code to the main
repository.

#26

Updated by Ivan Tcholakov almost 10 years ago

10111:fb816da9c3dd Feature #272 - A new function, api_substr_count() has been added to the internationalization library.
10110:a3bed3984f9e Feature #272 - Avoiding direct use of the function mb_convert_encoding(), replacing with api_convert_encoding().

#27

Updated by Ivan Tcholakov almost 10 years ago

10119:3331b2c21fae Feature #272 - A new implementation of pdf-export in the Wiki tool using mPDF 4.2 library.
10118:8e6c5b1f7764 Feature #272 - Adapting the library mPDF 4.2 to the system: Making the library independent from mbstring. Replacing "mb_" functions with functions from the internationalization library.
10117:5fa7d45fd751 Feature #272 - mPDF: When an image is referred by URL inside a course, the security system blocks reading it. A workaround: Let us use system paths for images when it is possible.
10116:8a27215ed4a4 Feature #272 - Adapting the library mPDF 4.2 to the system: Moving some directories for temporary files in the directory ".../chamilo/arcive/" where access for writting is guaranteed.
10115:2e702b2aa3d6 Feature #272: The library mPDF: Some directories and files have been removed, they are not needed.
10114:e92d4a25e23c The library mPDF v.4.2 (http://mpdf.bpm1.com/) has been imported, "as-is", the core files only, no additional fonts.
10113:b837e9ef90a8 Feature #272 - A new function, api_get_text_direction() has been added to the internationalization library. The online editor: A minor correction about determing text direction from language.
10112:f5d06747dfc9 Feature #272 - Fixing an author's name.

#28

Updated by Juan Carlos Raña Trabado almost 10 years ago

I've done some testing and I think the new library for pdf has many options to stay as the only pdf library. However, we must be prudent, to try it more time and see the views of others developers.

#29

Updated by Ivan Tcholakov almost 10 years ago

A message to Yannick Warnier.

Dear Yannick,

As I can see, a new translation tool is under development. If this new tool is under GNU/GPL license, I would like to see its code (when it is ready) in some public repository. Maybe I have something useful to say about it.

Ivan.

#30

Updated by Ivan Tcholakov almost 10 years ago

10130:996c2d9b86c7 Feature #272 - Gradebook tool, competences report: The PDF-exporting feature has been reworked to use the library mPDF, which is UTF-8 compatible. A nice, non-intentional effect: the generated report supports score-coloring if it was activated.
10129:5a482a966aa5 Feature #272 - The Gradebook: More formatting for a file before changing it.
10128:7f1eb89a2f80 Feature #272 - The Gradebook: Preparing a file for changes, formatting.

#31

Updated by Ivan Tcholakov almost 10 years ago

An image has been attached, it is about the reworked "Competences report".

#32

Updated by Ivan Tcholakov almost 10 years ago

In the Gradebook tool there are mode ezpdf-based pdf-exports. I can not see how do they work. It seems to me code is not active. I won't rework these (latent?) code fragments for UTF-8 compatibility by using mPDF. Let somebody more familiar with the Gradebook tool do this work, by following the example I did.

#33

Updated by Ivan Tcholakov almost 10 years ago

10133:f00cb03b9390 Feature #272 - Various improvements and unification about dealing with the system encoding, the text direction, the interface language.
10132:cfd66bf84313 Feature #272 - The internationalization library: A minor change about the function api_get_language_isocode(), a new optional parameter has been added.
10131:73ab7c5a7cab Feature #272 - A minor change, an expired comment has been removed.

#34

Updated by Ivan Tcholakov almost 10 years ago

10138:cbf070029903 Feature #272 - The PHP function fgetcsv() depends on the OS locale setting. When the file system encoding is different than the encoding of the CSV-file, then the imported data is damaged. For solving this problem, two new functions have been introduced in the internationalization library - api_str_getcsv() and api_fgetcsv().
10134:b6713357b933 Feature #272 - The Assignments tool: Fixing user's name ordering inside an e-mail sending routine.

#35

Updated by Ivan Tcholakov almost 10 years ago

10143:55c529d3bbb2 Feature #272 - The php function fgetcsv() has been replaced by api_fgetcsv(), see the internationalization library. The replacement function is binary safe, it does not depend on OS locale settings.

#36

Updated by Ivan Tcholakov almost 10 years ago

The next problem to be solved is about course archives.

The scenario
------------------

1.Let us imagine that an organization uses a stable system like Chamilo 1.8.6.2 or Dokeos 1.8.6.1. The platform character set is ISO-8859-15. Teachers have their course (training) arcives which are ISO-8859-15 encoded internally.

2. Chamilo 1.8.7 comes some day, is is UTF-8 by default, and our organization wants UTF-8 too. Ok, then our course archives should be importable in the new system without character encoding issues.


I will rework the Restore tool only in order it to be able to read/detect the internal encoding of the imported course archive. Then, the Restore tool would recode (if it is needed) all the string fields of the objects read to the proper (system) encoding.

I repeat, only the course importing feature should be slightly reworked. The rework is not radical, I am starting it in my place.

#37

Updated by Ivan Tcholakov almost 10 years ago

A report by aboivin, to be processed under this task.
http://www.chamilo.org/en/node/147

#38

Updated by Ivan Tcholakov almost 10 years ago

10156:fc41a03c42eb Feature #272 - Course restorer: Cleaning line endings in the source file.
10155:9249fa1a7b57 Feature #272 - Course restorer: Encoding detection for course archives has been added. It is necessary for importing archives that are created with systems older than 1.8.6.1. Next to do: The read/detected encoding value should be used for data conversion.
10151:b6134f04cf3f Feature #272 - A new customized function api_html_to_text() for stripping html-tags has been added in the internationalization library.
10148:631171b8e6d2 Feature #272 - A comment has been updated in the internationalization library.

#39

Updated by Ivan Tcholakov almost 10 years ago

10169:c49659405cb8 Feature #272 - Course restorer: Now the imported course is converted to the system encoding. For example, in UTF-8 system you may import old 1.8.x archives that have ISO-8859-15 encoding.

#40

Updated by Ivan Tcholakov almost 10 years ago

See the topic http://www.chamilo.org/en/node/147 . It initiated the following changes.
10175:d8c448417d5e Feature #272 - Fixing sorting by dates in the Documents tool.
10174:58ca456c9f8a Feature #272 - Fixing sorting by dates in the Assignments tool. An initial workaround has been proposed by André Boivin (aboivin), see http://www.chamilo.org/en/node/147
10172:d0a1919a80b1 Feature #272 - Reworks for the TableSort class: Adding functionality for parsing hidden raw date values for maikng precise sorting by dates. Elimination of useless recursion. Simplifying the constructed callback function for sorting.

#41

Updated by Ivan Tcholakov almost 10 years ago

10260:d62ec912ddf5 Feature #272 - The old online conferention tool: Minor corrections in a html header for declaring language, encoding and text direction.
http://code.google.com/p/chamilo/source/detail?r=d62ec912ddf53a0d60b20e8dcee1ad960f7dbe89&repo=classic

10259:abc21a9f4bad Feature #272 - Code update according to the recent changes within the Database class - direct use elimination in some places of MySQL-specific functions.
http://code.google.com/p/chamilo/source/detail?r=abc21a9f4bad7165201d86d7303fa019ac132220&repo=classic

10258:e783ccf6b356 Feature #272 - Updating the Database class testing unit.
http://code.google.com/p/chamilo/source/detail?r=e783ccf6b3567db80b1795dfa4441f7e88b1b14e&repo=classic

10257:9abca585bd86 Feature #272 - A minor correction in the Security class.
http://code.google.com/p/chamilo/source/detail?r=9abca585bd864003950e8bdda08f147808b9009c&repo=classic

10256:21737b4beebc Feature #272 - The Database class has been reworked and upgraded in order some of its common methods to deal with multiple database connections simultaneously.
http://code.google.com/p/chamilo/source/detail?r=21737b4beebcc74a493e0ada83b6c1d64c33a1bf&repo=classic

10255:63adfc44f2fc Feature #272 - The function replace_dangerous_char() has been updated, the file name encoding detection inside it has been implemented through using a special function from the internationalization library.
http://code.google.com/p/chamilo/source/detail?r=63adfc44f2fcec252a1b9e49424aae530e4ff353&repo=classic

#42

Updated by Ivan Tcholakov almost 10 years ago

10262:26d27fa61127 Feature #272 - Elimination of mysql_*() function calls.
http://code.google.com/p/chamilo/source/detail?r=26d27fa611274183d9541a02f8d6dc57ad0f0ad5&repo=classic

10261:b159ded90119 Feature #272 - Two common purpose methods have been added to Database class.
http://code.google.com/p/chamilo/source/detail?r=b159ded901199941226b478cf954c5056ae451ec&repo=classic

#43

Updated by Ivan Tcholakov almost 10 years ago

About the question from Yannick in http://code.google.com/p/chamilo/source/detail?r=21737b4beebcc74a493e0ada83b6c1d64c33a1bf&repo=classic

In addition to my previous answer, I am attaching here a scheme to show the idea in easy way.

--------------------------------- Update 11-FEB-2010 ---------------------------------------
After digging in the code of the installation scripts I found that the implementation that
is needed does not comply to the picture I've sent here. What is needed is simpler - (some)
tables should be examined about their encoding and collation properties. Then the database
server will recode the existing tables.

If it is necessary, a special, non-standard case could be detected and processed too - when
in the old system UTF-8 data is saved into ISO-88559-1 encoded tables.

#44

Updated by Ivan Tcholakov almost 10 years ago

10276:ab61d8c83421 Feature #272 - In the transaction 26d27fa61127 some "silent" sentences die() were created. Here the previous behaviour die(Database::error()) has been restored.
http://code.google.com/p/chamilo/source/detail?r=ab61d8c83421cc8862fc086ee6373627acbbad43&repo=classic

10275:6fa44338e45e Feature #272 - Installation scripts: Cleaning, formatting - preparation for some essential changes.
http://code.google.com/p/chamilo/source/detail?r=6fa44338e45ef317d9b10953131eec475b762eae&repo=classic

10268:f051e44ad553 Feature #272 - The installation script: Some minor corrections and cosmetic changes.
http://code.google.com/p/chamilo/source/detail?r=f051e44ad553c131c611e1dd3f94ee2de42c496d&repo=classic

#45

Updated by Ivan Tcholakov almost 10 years ago

10286:39941fd5d1b7 Feature #272 - Adding the method Database::get_tables().
http://code.google.com/p/chamilo/source/detail?r=39941fd5d1b7c267bdc7dc2dedafbf0b1ae7c185&repo=classic

10282:e3c596655705 Feature #272 - About the database connection testing button: The word "MySQL" on the page has been replaced with "Database".
http://code.google.com/p/chamilo/source/detail?r=e3c5966557052f97fe02ad6637b5519c166e67da&repo=classic

10280:03cad60198a6 Feature #272 - Adding a new method Database::get_databases().
http://code.google.com/p/chamilo/source/detail?r=03cad60198a662f06ac3e452131d7dec5155a213&repo=classic

10279:c9e6e06060b1 Feature #272 - Installation scripts: Corrections on some html-tags.
http://code.google.com/p/chamilo/source/detail?r=c9e6e06060b1c5f59e01e2186bb7f69e6c986988&repo=classic

10278:aaea3b166d7f Feature #272 - Applying the function api_get_path() in the tnstallation scripts. The function has been designed for being capable about serving the installation procedure.
http://code.google.com/p/chamilo/source/detail?r=aaea3b166d7fa505b81a55be1bbaf9427f6b54ca&repo=classic

10277:7cd544892006 Feature #272 - The installation scripts: An easy rework about the database connection testing button.
http://code.google.com/p/chamilo/source/detail?r=7cd5448920061fd4b7db1bfa2a7568f5ee7d9e8f&repo=classic

#46

Updated by Ivan Tcholakov almost 10 years ago

10294:aa6db2603a33 Feature #272 - Adding a new method Database::get_fields().
http://code.google.com/p/chamilo/source/detail?r=aa6db2603a334cc4c50952f7908b668a882f2f0c&repo=classic

#47

Updated by Ivan Tcholakov almost 10 years ago

10296:10e53b5545df Feature #272 - Extending functionality of the method Database::get_fields().
http://code.google.com/p/chamilo/source/detail?r=10e53b5545df29c2f01bc715b09b46edf62b6568&repo=classic

#48

Updated by Ivan Tcholakov almost 10 years ago

10336:75fcbf24db87 Feature #272 - Installation scripts: Cleaning & reformatting, the third pass.
http://code.google.com/p/chamilo/source/detail?r=75fcbf24db870af4e35d372186d4145feab8d667&repo=classic

10335:afc41667c8e7 Feature #272 - Installation scripts: Cleaning & reformatting, the second pass. Some minor corrections.
http://code.google.com/p/chamilo/source/detail?r=afc41667c8e7f98274d19571baf3bfda6e8f5cf0&repo=classic

#49

Updated by Ivan Tcholakov almost 10 years ago

10339:b3ac5516219b Feature #272 - Installation scripts: Some direct mysql_*() calls have been eliminated.
http://code.google.com/p/chamilo/source/detail?r=b3ac5516219b0f51959dc415469bda09fb69da69&repo=classic

10338:043bc080eb3b Feature #272 - The functions api_get_permissions_for_new_directories() and api_get_permissions_for_new_files() have been introduced in the installation scripts.
http://code.google.com/p/chamilo/source/detail?r=043bc080eb3b71c868b695c71c0c89dab9025e0b&repo=classic

10337:4d6ad81b2fcb Feature #272 - Providing for the installation scripts limited support for the function api_get_settings().
http://code.google.com/p/chamilo/source/detail?r=4d6ad81b2fcb9b92708c3931272a9b6e4fb0f06c&repo=classic

#50

Updated by Ivan Tcholakov almost 10 years ago

10341:ae24a8c497d5 Feature #272 - Fixing a bug in the internationalization library. The bug appeared when mbstring was not installed. The symptom: api_htmlentities() returned empty result when the input parameter wasn't string-type.
http://code.google.com/p/chamilo/source/detail?r=ae24a8c497d543310f5c5c7c2814f8801a12296c&repo=classic

10340:2231d50d8623 Feature #272 - Installation scripts: Testing new installation procedure, minor corrections.
http://code.google.com/p/chamilo/source/detail?r=2231d50d862365b109bdfd82187163d8687890ae&repo=classic

#51

Updated by Ivan Tcholakov almost 10 years ago

10342:e5c32fc312fb Feature #272 - Internationalization library: Reworks for some functions to make them stable against non-string input parameters.
http://code.google.com/p/chamilo/source/detail?r=e5c32fc312fbb2cd46e1ad100bbb0d40e4d2ffa6&repo=classic

#52

Updated by Ivan Tcholakov almost 10 years ago

10350:ee9b567aea75 Feature #272 - Installation scripts: Reworks, cleaning.
http://code.google.com/p/chamilo/source/detail?r=ee9b567aea75e022b1641ad003067c0667b8c6cc&repo=classic

10349:e4f4f29e8dcf Feature #272 - Installation scripts: Modernizing some code in install_db.inc.php.
http://code.google.com/p/chamilo/source/detail?r=e4f4f29e8dcf8ca792b1c521ae4f1d1b47658423&repo=classic

10345:886c33cd2f36 Feature #272 - Corrections for the function api_display_language_form().
http://code.google.com/p/chamilo/source/detail?r=886c33cd2f36424953c08e26515745107d9e6eeb&repo=classic

#53

Updated by Ivan Tcholakov almost 10 years ago

10354:d29f0de1aa1f Feature #272 - Installation scripts: More reworks about the language selection boxes.
http://code.google.com/p/chamilo/source/detail?r=d29f0de1aa1f98dca0cfa3ef51e083262e66bef5&repo=classic

10353:0b07c6509fe4 Feature #272 - Installation scripts: Code for a language selection box has been reworked. It is about language during installation procedure. Some other minor reworks.
http://code.google.com/p/chamilo/source/detail?r=0b07c6509fe44dcffb7c06492015d924c3d61043&repo=classic

#54

Updated by Ivan Tcholakov almost 10 years ago

10355:dbd9e0751856 Feature #272 - Installation scripts: A rework for the function get_config_param() for making it understandable.
http://code.google.com/p/chamilo/source/detail?r=dbd9e07518561538e66a902bfffee6248bb2f2f2&repo=classic

#55

Updated by Ivan Tcholakov almost 10 years ago

10365:1992f77a8ce4 Feature #272 - Installation scripts: If it is possible, the initial value of the language during the installation procedure is according to the the browser.
http://code.google.com/p/chamilo/source/detail?r=1992f77a8ce4c4f33083993ef7950d8dc0444dfc&repo=classic

#56

Updated by Ivan Tcholakov almost 10 years ago

  • % Done changed from 50 to 60

Ivan Tcholakov wrote:

10365:1992f77a8ce4 Feature #272 - Installation scripts: If it is possible, the initial value of the language during the installation procedure is according to the the browser.
http://code.google.com/p/chamilo/source/detail?r=1992f77a8ce4c4f33083993ef7950d8dc0444dfc&repo=classic

#57

Updated by Ivan Tcholakov almost 10 years ago

10430:88c8631503be Feature #272 - Installation scripts: All the deprecated files have been removed.
10429:987db0faf55d Feature #272 - Installation scripts: Making a switch to the new library file install.lib.php.
10428:a7515e6d1449 Feature #272 - Installation scripts: A new file install.lib.php has been introduced. It is to replace the files install_functions.inc.php and install_upgrade.lib.php.
10417:debffc12e0ea Feature #272 - Installation scripts: Recovering from changes made by mistake in the previous transaction in the file migrate-db-1.8.6.2-1.8.7-pre.sql.
10416:8166995b91ec Feature #272 - Installation scripts: More code formatting.
10398:ca105a880ef0 Feature #272 - Providing a better way for the function api_get_path() for detecting when installation runs, by using the constant SYSTEM_INSTALLATION.
10397:19350af19c3d Feature #272 - Installation scripts: Several files have been renamed before being finally deleted.
10396:cf128703d63e Feature #272 - Installation scripts: Some minor changes.
10395:6f950332fa3e Feature #272 - Installation scripts: Replacement of all mysql_*() function calls with Database::*() equivalents.
10394:77b16578eb41 Feature #272 - Installation scripts: Replacement of all ereg*() function calls with preg*() equivalents.

#58

Updated by Ivan Tcholakov almost 10 years ago

10432:de204824ef54 Feature #272 - Installation scripts: Repetitive code elimination.
http://code.google.com/p/chamilo/source/detail?r=de204824ef549946ebf6ca5451a5bf209f90b62f&repo=classic

#59

Updated by Ivan Tcholakov almost 10 years ago

10457:aee28cee0475Feature #272 - Installation scripts: Duplicate code elimination.
http://code.google.com/p/chamilo/source/detail?r=aee28cee04756887f8d7ea0c2041d36d2705ee73&repo=classic

10456:ef519c35933aFeature #272 - Installation scripts: Updating comments.
http://code.google.com/p/chamilo/source/detail?r=ef519c35933aa3ae513d7024cae4c9b9b474743e&repo=classic

#60

Updated by Ivan Tcholakov almost 10 years ago

10474:399d9cd1671c Feature #272 - Installation scripts: Reorganization of the library install.lib.php. Other minor changes.
http://code.google.com/p/chamilo/source/detail?r=399d9cd1671c329cb79030dec204f950b1c5d1a1&repo=classic

#61

Updated by Ivan Tcholakov almost 10 years ago

10482:a8494999cec7 Feature #272 - Installation scripts: Renaming files, removing the old brand.
http://code.google.com/p/chamilo/source/detail?r=a8494999cec7a14b40eafd6e4ea357dffcda7b07&repo=classic

#62

Updated by Ivan Tcholakov almost 10 years ago

10491:2621ce8914da Feature #272 - Installation scripts: Removing a duplicate setting.
http://code.google.com/p/chamilo/source/detail?r=2621ce8914da51ec263dcf6f3da452f9354829a4&repo=classic

10490:bdd1356f578f Feature #272 - Installation scripts: UTF-8 database migration has been implemented in the upgrading script.
http://code.google.com/p/chamilo/source/detail?r=bdd1356f578fd30dd591098e6a298b49ee6a4ba4&repo=classic

10489:22a0b53c9764 Feature #272 - Installation scripts: Reverting my customized code that was inserted by mistake.
http://code.google.com/p/chamilo/source/detail?r=22a0b53c97640e8791e5a301ab70e78d5a33d712&repo=classic

#63

Updated by Ivan Tcholakov almost 10 years ago

  • % Done changed from 60 to 80

10504:b856b6fb3927 Feature #272 - Adding a simple protection against re-installing the system.
http://code.google.com/p/chamilo/source/detail?r=b856b6fb3927ba8af3ef9acdc7db5a754954e5bc&repo=classic

See the related forum topic http://www.chamilo.org/en/node/195

10492:c691bdcaac05 Feature #272 - Temporay workarounds (fixing wrong syntax) for English, Spanush and Bulgarian language files.
http://code.google.com/p/chamilo/source/detail?r=c691bdcaac05ffba74d10d494db8e85ef20d6cb9&repo=classic

#64

Updated by Yannick Warnier almost 10 years ago

Ivan Tcholakov wrote:

A message to Yannick Warnier.

Dear Yannick,

As I can see, a new translation tool is under development. If this new tool is under GNU/GPL license, I would like to see its code (when it is ready) in some public repository. Maybe I have something useful to say about it.

Ivan.

Hi Ivan,

You can download the new translation tool code from the Google repository (and you can send patches as well). Basically, it is in the default Chamilo 2 code. You have to clone the "chamilo" repository, and then install it with only the "translation" application (you will find that in the install process). Sven and Hans of the 2.0 team are responsible of this, but I have sent a few patches today. You have commit access too.
Look into the application/lib/cda/ directory for the translation app code.

#65

Updated by Ivan Tcholakov almost 10 years ago

Thank you.

10549:3c82709d2d39 Feature #272 - Updating the language files for testing pure UTF-8 files on the system. Still, these files are created with a special script, outside the CDA.
http://code.google.com/p/chamilo/source/detail?r=3c82709d2d39b482eb902ac0523ceafc7221870a&repo=classic

10548:c4dfb6251a27 Feature #272 - Removing new language folders that didn't have the same names as the old folders have.
http://code.google.com/p/chamilo/source/detail?r=c4dfb6251a27e814df32fb5ba0f2abeba2521472&repo=classic

10547:97b9fc34da04 Feature #272 - A minor change in a test for the internationalization library.
http://code.google.com/p/chamilo/source/detail?r=97b9fc34da0426a6d1dad219eed55d9ac67b3548&repo=classic

10538:b203107b7123 Feature #272 - Installation scripts: A unused language-related constant has been removed.
http://code.google.com/p/chamilo/source/detail?r=b203107b7123f93e48ed529b0f69473c3dc363c6&repo=classic

10536:14e69dc97115 Feature #272 - Changing a default value ISO-8859-15 ---> UTF-8 in the global initialization script.
http://code.google.com/p/chamilo/source/detail?r=14e69dc97115f1ec53c8c29d9c0c1f63f2f7c969&repo=classic

10530:588b5941d72b Feature #272 - An insignificant change in the internationalization library.
http://code.google.com/p/chamilo/source/detail?r=588b5941d72bd3a1238c51213ccfebd85006905c&repo=classic

#66

Updated by Ivan Tcholakov almost 10 years ago

10557:ff2281034605 Feature #272 - A revision of the native language names, the idea is from Chamilo 2.0. Source used: http://cldr.unicode.org/index/downloads/cldr-1-7-2
http://code.google.com/p/chamilo/source/detail?r=ff2281034605705c8cf6a6cb1b721675dcfa2841&repo=classic

#68

Updated by Ivan Tcholakov almost 10 years ago

  • % Done changed from 80 to 90
#69

Updated by Julio Montoya almost 10 years ago

  • % Done changed from 90 to 80

I think that the feature that prevent the re-installation of Chamilo should be able only if my server is in "Production mode".

#70

Updated by Julio Montoya almost 10 years ago

  • % Done changed from 80 to 90

I test the migration from Dokeos 1.8.6.1 to Chamilo 1.8.7 And it does not work well.

I received a warning message here:

install/update-files-1.8.6.1-1.8.6.2.inc.php

// The following line is disabled, connection has been already done
//$link = Database::connect(array('server' => $dbHostForm, 'username' => $dbUsernameForm, 'password' => $dbPassForm));
//Database::select_db($dbNameForm, $link);
Database::select_db($dbNameForm);
#71

Updated by Ivan Tcholakov almost 10 years ago

In the installation scripts there is no "Production mode". This setting exists is the database of already installed system.
About the warning message - I will make tests to see what the problem is.

#72

Updated by Julio Montoya almost 10 years ago

That setting exists when you upgrade Chamilo, that what I meant.

Ivan Tcholakov wrote:

In the installation scripts there is no "Production mode". This setting exists is the database of already installed system.

#73

Updated by Ivan Tcholakov almost 10 years ago

Ok, if you say so. But "Production/test mode" was introdused in the past for developer's convenience. Error reporting level changes, database errors are shown, etc.
Upgrading is made by the end-user (mostly), so "Production/test mode" is irrelevant to this case.

I think, the design of the re-installation protection feature is good enough. I only will test it again to see whwther it works as I expect.

#74

Updated by Julio Montoya almost 10 years ago

It was just a suggestion what I said.
Here (in Lima) we test the upgrading process several times before every released (1.8.6, 1.8.5, 1.8.4, etc).
I think is relevant to improve productivity in our tests. (Run migration process several times) if I have this feature blocking the migration,is a little bit annoying.

Ivan Tcholakov wrote:

Ok, if you say so. But "Production/test mode" was introdused in the past for developer's convenience. Error reporting level changes, database errors are shown, etc.
Upgrading is made by the end-user (mostly), so "Production/test mode" is irrelevant to this case.

I think, the design of the re-installation protection feature is good enough. I only will test it again to see whwther it works as I expect.

#75

Updated by Ivan Tcholakov almost 10 years ago

I explained how the blocker works here: http://www.chamilo.org/en/node/195 . During the upgrade the script should not block (it compares versions).

#76

Updated by Ivan Tcholakov almost 10 years ago

Could you explain in details what "Run migration process several times" is, this sounds new to me.

#77

Updated by Ivan Tcholakov almost 10 years ago

You may temporarily disable the blocking feature, for example:

install/index.php, line 140

if (is_already_installed_system()) {

change it to be

if (false) {

and the blocker will be disabled for the tests. Only you should be careful this change not to go inside the repository.

#78

Updated by Julio Montoya almost 10 years ago

Thanks for the advice

Ivan Tcholakov wrote:

You may temporarily disable the blocking feature, for example:

install/index.php, line 140

if (is_already_installed_system()) {

change it to be

if (false) {

and the blocker will be disabled for the tests. Only you should be careful this change not to go inside the repository.

#79

Updated by Ivan Tcholakov almost 10 years ago

Important, for Julio: A correction in the repository's code is needed for making upgrading possible. I've just done it.

10685:8947e216ddbf Task #272 - Installation scripts, upgrading: Forsing inclusion of the old configuration file. This is needed to be done after the blocking feature has been introduced.
http://code.google.com/p/chamilo/source/detail?r=8947e216ddbf101b362396afba0ef04488725479&repo=classic

#80

Updated by Julio Montoya almost 10 years ago

Thanks again :)

#81

Updated by Ivan Tcholakov almost 10 years ago

Julio, here is yet another transaction about the upgrade scripts:

10687:4dbfaa1bd4c9 Task #272 - Installation scripts, upgrading: Adding a work of mine that was unfinished.
http://code.google.com/p/chamilo/source/detail?r=4dbfaa1bd4c94a45113af540fb6cc1f3ac9c16b0&repo=classic

Sorry, I did not have time to finish this work earlier. This is it.

#82

Updated by Ivan Tcholakov almost 10 years ago

10696:083a0aa02faa Feature #272 - A minor correction for platform settings page.
http://code.google.com/p/chamilo/source/detail?r=083a0aa02faaa72b39cdf5d7a64891593cd58efa&repo=classic

For tomorrow: On clean installation we have to make sure that the selected by the administrator platform language is enabled in the newly produced database settings.

#83

Updated by Ivan Tcholakov almost 10 years ago

10699:cd8ecd76df27 Feature #272 - Portal homepage: Making sure that html-snippets that come from files will always be shown using the system encoding.
http://code.google.com/p/chamilo/source/detail?r=cd8ecd76df27dd0874061aa28dfae6c1d34db3d6&repo=classic

10698:b87e825c7d06 Feature #272 - A specific statement has been made to process https addresses too.
http://code.google.com/p/chamilo/source/detail?r=b87e825c7d067fe69d3a8f97885e15a22d5dc0bd&repo=classic

10697:4b057bd4e304 Feature #272 - Minor changes in the internationalization library.
http://code.google.com/p/chamilo/source/detail?r=4b057bd4e3045b779574345705632960a2627817&repo=classic

#84

Updated by Ivan Tcholakov almost 10 years ago

10720:d3975ef41149 Feature #272 - "Configure homepage": Reworks for managing encodings of the included html-snippets.
http://code.google.com/p/chamilo/source/detail?r=d3975ef41149e8ebdf907561a470dc5d038e94ff&repo=classic

#85

Updated by Ivan Tcholakov almost 10 years ago

10747:4466dee74c16 Feature #272 - "Registration" and "Registration settings": Adding code for managing possible encoding differencies between the system and the stored in files html-snippets.
http://code.google.com/p/chamilo/source/detail?r=4466dee74c1643534ca8bf2424fe96e1b034d2b9&repo=classic

#86

Updated by Ivan Tcholakov almost 10 years ago

10753:9ac7dcd26f5e Feature #272 - The banner: Adding code for managing possible encoding differencies between the system and the stored in files html-snippets. Minor changes in "Configure homepage".
http://code.google.com/p/chamilo/source/detail?r=9ac7dcd26f5e142b816fbfc8b0d8b918cfa98851&repo=classic

#87

Updated by Ivan Tcholakov almost 10 years ago

10756:cd50bd36f8c8 Feature #272 - Showing plain text files: Passing encoding detection result to the browser.
http://code.google.com/p/chamilo/source/detail?r=cd50bd36f8c858930bbfbf801fb9301c8c159d78&repo=classic

#88

Updated by Ivan Tcholakov almost 10 years ago

10757:63c0d8601274 Feature #272 - Fixing the function api_get_local_time() for non-UTF-8 systems on PHP 5.3 The "intl" extension works with UTF-8 encoding only (good), but the Chamilo system could use ISO-8859-1, WINDOWS-1251, ...
http://code.google.com/p/chamilo/source/detail?r=63c0d860127415aa3aa80effc643c4520a391393&repo=classic

A message to Guillaume Viguier: Good work. On the system there are some obsolete date-about routines. I tried to write in the internationalization library something about dates, but it stays like unfinished. For 1.8.7.1 we could clean some obsolete stuff.

#89

Updated by Ivan Tcholakov almost 10 years ago

A picture has been attached to the previous message.

#90

Updated by Ivan Tcholakov almost 10 years ago

10760:fbc4d2793223 Feature #272 - A correction for the previous transaction.
http://code.google.com/p/chamilo/source/detail?r=fbc4d2793223c7b074ae7561c4b7ce47218f97ed&repo=classic

10759:23bbb5b68b3c Feature #272 - HTML files: Passing read/detected encoding value to the browser using the HTTP-header (too). Thus, not well-formed html-files have good chance to be showed using the right encoding.
http://code.google.com/p/chamilo/source/detail?r=23bbb5b68b3cfa8cfde97c96fb755fb115ebaba5&repo=classic

10758:37cd0e5c9739 Feature #272 - A new function api_detect_encoding_html() has been added in the internationalization library. A test for this function has been created.
http://code.google.com/p/chamilo/source/detail?r=37cd0e5c9739162d9741615d31ea48d643382911&repo=classic

#91

Updated by Ivan Tcholakov almost 10 years ago

10763:ca497cf865d3 Feature #272 - The Dropbox tool fully uses the conventional language sub-system, the function dropbox_lang() has been eliminated.
http://code.google.com/p/chamilo/source/detail?r=ca497cf865d3754e8352b56cd82d00b23c6ca788&repo=classic

#92

Updated by Ivan Tcholakov almost 10 years ago

10800:eaecad6c4b18 Feature #272 - The Dropbox tool code - additional minor changes.
http://code.google.com/p/chamilo/source/detail?r=eaecad6c4b186f4cae59045866aba3588449c7d7&repo=classic

10799:b7dd21966ab3 Feature #272 - The Dropbox tool code - cosmetic changes before abandonning it.
http://code.google.com/p/chamilo/source/detail?r=b7dd21966ab36f96c487b73a9f80ec6aa9a760b7&repo=classic

#93

Updated by Ivan Tcholakov almost 10 years ago

TODO: Dropbox and Assignments: To be checked whether the function replaced_dangerous_char() is used everywhere it should be used. How to test: Files with localized names should be accepted with renaming them using transliteration to ASCII letters only.

#94

Updated by Ivan Tcholakov almost 10 years ago

10811:5f03846da9fd Feature #272 - Some corrections in "Assignments" tool. Code cleaning for "Assignments" and "Dropbox" tools.
http://code.google.com/p/chamilo/source/detail?r=5f03846da9fd12862c670d5a565d2bf5e1862bed&repo=classic

A note: In the "Assignments" tool there is copy/pasted code from "Documents" tool. Not all of the copied code is needed probably, but I have no time for analysis.

#95

Updated by Ivan Tcholakov almost 10 years ago

Hmm..., since database has moved to UTF-8, we need to increase the length of some text fields. This is related for newly installed systems and for newly created courses.

Using the upgrading script, while converting to UTF-8 MySQL server increases automatically length of text-type fields.

See http://dev.mysql.com/doc/refman/5.1/en/alter-table.html
especially the explanations after the example:
ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name;

There are two options:
1. For a newly installed system we to make the same as MySQL server makes automatically for all the text fields.
2. Selectively to increase some text fields, especially that are related to "title" and "description" meanings. I noticed this necessity for Bulgarian language. Text-fields that contain ASCII keywords used by the system should not change their length.

The second option is preferable by me. I hope, I'll manage before the 1.8.7 beta release to prepare a proposal.

#96

Updated by Ivan Tcholakov almost 10 years ago

10812:de05e885277d Feature #272 - Edition of comments in "Dropbox" tool. Function simplification in "Assignments" tool.
http://code.google.com/p/chamilo/source/detail?r=de05e885277d0a8efa84a3d86c157b7e66799710&repo=classic

#97

Updated by Ivan Tcholakov almost 10 years ago

10814:79d4412542d2 Feature 272 - Applying the function replace_dangerous_char() for file names of downloaded files. Cosmetic changes in the main API.
http://code.google.com/p/chamilo/source/detail?r=79d4412542d258663b983f82cebe3a43e56bdcf3&repo=classic

#98

Updated by Ivan Tcholakov almost 10 years ago

10818:446ac09c72ab Feature #272 - A workaround: Creation of a new function for formatting long dates, it should work on PHP 5.2.x and PHP 5.3+.
http://code.google.com/p/chamilo/source/detail?r=446ac09c72ab8a8f2e20cae23030412fff8d0fe0&repo=classic

#99

Updated by Ivan Tcholakov almost 10 years ago

10819:fce6f58c9ee4 Feature #272 - The internationalization library: Making the date/time formatting functions more convenient for use.
http://code.google.com/p/chamilo/source/detail?r=fce6f58c9ee490e7f041b9ca56ae4ae718bf9f54&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

10821:a5861bcc8d60 Feature #272 - "Assignments" and "Dropbox" tools: Showing dates correctly on PHP 5.2 and PHP 5.3.
http://code.google.com/p/chamilo/source/detail?r=a5861bcc8d608fdbeecc4d7f4ba64002eab4a284&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

10831:b10b13c30e13 Feature #272 - "Dropbox" tool: Fixing a mistake.
http://code.google.com/p/chamilo/source/detail?r=b10b13c30e130aab771a7b8bce8db0be98d03674&repo=classic

Thank you again Guillaume, for noticing this mistake.

Updated by Ivan Tcholakov almost 10 years ago

10833:c00ed614c6d5 Feature #272 - Cleaning of the file text.lib.php. Making the function date_to_str_ago() more conveniet for use.
http://code.google.com/p/chamilo/source/detail?r=c00ed614c6d57653c7689436a01a21e33dfb7879&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

10847:a7edd8674be3 Feature #272 - Revision and cleaning of all the include files.
http://code.google.com/p/chamilo/source/detail?r=a7edd8674be373993b72bf4f1a7543fa09a885a2&repo=classic

10846:016e622e3aab Feature #272 - "Dropbox" tool, downloading zip-archives: Adding html-metadata to the generated supplemental file "overview.html" (which you may find inside the downloaded archive).
http://code.google.com/p/chamilo/source/detail?r=016e622e3aab4ef0229e57647c92d0857548484e&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

Here is my proposition for expanding some database text fields. The goal is titles, descriptions, etc. not to be truncated due to low length of some fields.

The proposed new values are based on my experience with a real Bulgarian language course on a UTF-8 system. Maybe I did not detected all the fields to be extended, this could be done if somebody complains.

I am requesting to implement these changes.


Database                      Table                Field, current definition                 Field, new definition
----------------------------------------------------------------------------------------------------------------------------------

chamilo_main                  course_module        name varchar(100) NOT NULL                name varchar(255) NOT NULL
(see db_main.sql)
                              sys_calendar         title varchar(200) NOT NULL               title varchar(255) NOT NULL

                              tag                  tag char(80) NOT NULL                     tag varchar(255) NOT NULL

chamilo_coursecode            quiz                 title varchar(200) NOT NULL               title varchar(255) NOT NULL
(see add_course.lib.inc.php)  quiz                 sound varchar(50) default NULL            sound varchar(255) default NULL

                              quiz_question        question varchar(200) NOT NULL            question varchar(511) NOT NULL

                              tool                 name varchar(100) NOT NULL                name varchar(255) NOT NULL
                              tool                 image varchar(100) default NULL           image varchar(255) default NULL
                              tool                 admin varchar(200) default NULL           admin varchar(255) default NULL
                              tool                 address varchar(120) default NULL         address varchar(255) default NULL

                              calendar_event       title varchar(200) NOT NULL               title varchar(255) NOT NULL

                              student_publication  url varchar(200) default NULL             url varchar(255) default NULL
                              student_publication  title varchar(200) default NULL           title varchar(255) default NULL
                              student_publication  description varchar(250) default NULL     description varchar(255) default NULL
                              student_publication  author varchar(200) default NULL          author varchar(255) default NULL

                              lp                   name tinytext not null                    name varchar(255) not null

                              lp_item              title tinytext not null                   title varchar(511) not null
                              lp_item              description tinytext    not null default ''  varchar(511) not null default ''

Updated by Ivan Tcholakov almost 10 years ago

10960:0b33ca548208 Feature #272 - The chosen during the installation platform language should be enabled for sure, automatically.
http://code.google.com/p/chamilo/source/detail?r=0b33ca54820874efe3a5f59bf0d3330868175a6d&repo=classic

A suggestion:

When the table "languages" is created and filled, all the languages may be disabled (or maybe only English to be left enabled). After the installation, the admin will have platform language enabled only and he/she will not be forsed to disable manually languages that are not needed.

Saying in another way: You install the system and you get only one language enabled - the chosen platform language. If you want additional languages, then use the correspondent administrator's page.

I would implement this suggestion on approval only.

Updated by Ivan Tcholakov almost 10 years ago

10962:3a0ea45539cf Feature #272 - Chat tool: Applying coding conventions and cleaning before making essential changes.
http://code.google.com/p/chamilo/source/detail?r=3a0ea45539cfe3f7e55a12c8544ddfce292eed02&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

10964:98b528e9f31c Feature #272 - Chat tool, frames: Adding html-instructions about encoding, language and text direction.
http://code.google.com/p/chamilo/source/detail?r=98b528e9f31c02acea3bd01a8583c1665c73658b&repo=classic

10963:7d4c3e8a137b Feature #272 - Chat tool, frames: Adding html-instructions about language and text direction.
http://code.google.com/p/chamilo/source/detail?r=7d4c3e8a137bb95488a49688b67b33395439ea8b&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

10968:e41a2cab4c95 Feature #272 - "Groups" tool: Minor corrections and code cleaning.
http://code.google.com/p/chamilo/source/detail?r=e41a2cab4c958bb1f305481b2b7dca5cacc59ec9&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

10976:92c238eb6b52 Feature #272 - fileManage.lib.php and text.lib.php: Revision, coding conventions, cleaning.
http://code.google.com/p/chamilo/source/detail?r=92c238eb6b523f42131de0687fc88384bf8d769d&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

10978:0711614256f9 Feature #272: The library fileUpload.lib.php: Revision, code conventions, cleaning, removing the deprecated functions replace_accents() and remove_accents().
http://code.google.com/p/chamilo/source/detail?r=0711614256f9611ec32cb78f7e6202eb7c0e8ae2&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

10979:fae89008aa09 Feature #272 - The "Documents" tool, part 1: Revision, code conventions and cleaning.
http://code.google.com/p/chamilo/source/detail?r=fae89008aa091e365516cca04913b167983a2976&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

10980:af961a4c0e97 Feature #272 - The "Documents" tool, part 2: Revision, code conventions and cleaning.
http://code.google.com/p/chamilo/source/detail?r=af961a4c0e971b0cf19d800a12e991a9a3dc7e70&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

10981:e636232e20bd Feature #272 - The "Documents" tool, part 3: Revision, code conventions and cleaning.
http://code.google.com/p/chamilo/source/detail?r=e636232e20bd5c3a328e3737f6f9ca19f43e48ec&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

10982:2abd12cf3ccf Feature #272 - The main API: Revisiting some of old TODOs of mine + cleaning.
http://code.google.com/p/chamilo/source/detail?r=2abd12cf3ccf47256604ca278f0becc15121169a&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

11012:9af6082eb1ad Feature #272 - Course copy: Setting encoding has been added during the initialization of Course class.
http://code.google.com/p/chamilo/source/detail?r=9af6082eb1add156911eba86ddf19473a385b6bc&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

11018:cd341c4b7424 Feature #272 - PEAR, HTML_Common class: Upgrading to version 1.2.5 (last stable). There is a solution inside about the function htmlspecialchars() and the encoding it uses as it is called within PEAR.
http://code.google.com/p/chamilo/source/detail?r=cd341c4b7424e15123e3e3f5ee222d75b141f28b&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

11019:7581146bda35 Feature #272 - PEAR, HTML_Table class: Upgrading from version 1.7.3 to version 1.8.2 (last known stable).
http://code.google.com/p/chamilo/source/detail?r=7581146bda35cbde57f45dbfd1af9f320cad421e&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

11021:97268fa48587 Feature #272 - Upgrading the basic PEAR class form version 1.4.11 to version 1.9.0.
http://code.google.com/p/chamilo/source/detail?r=97268fa48587ad5a6ef71ddcf8f69039974b7077&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

11022:530e261e7aaa Feature #272 - PEAR, Quick_Forms: Preparation for a upgrade, the current version has been identified as 3.2.7. Making code to be close to the original source, Chamilo's customizations have been distinguished.
http://code.google.com/p/chamilo/source/detail?r=530e261e7aaab17de00abe2bda957b4b1e926bcb&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

11024:a11c8e0e7500 Feature #272 - PEAR, HTML_QuickForm_Controller package: Prepararion for upgrade. The current version is 1.3.
http://code.google.com/p/chamilo/source/detail?r=a11c8e0e750074038633bedcecdb436f1949c8a4&repo=classic

11023:7b1bfd5c731c Feature #272 - PEAR, HTML_QuickForm_advmultiselect class: Prepararion for upgrade. The current version is 0.5.1.
http://code.google.com/p/chamilo/source/detail?r=7b1bfd5c731c853971485c968518ca9e6948088e&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

11025:20b9e7b7cb70 Feature #272 - PEAR, Pager class: Prepararion for upgrade. The current version is 2.4.2.
http://code.google.com/p/chamilo/source/detail?r=20b9e7b7cb703409f27607e42a228ae90b25efe1&repo=classic

Updated by Ivan Tcholakov almost 10 years ago

11026:1d2159e3cc28 Feature #272 - Upgrading some PEAR packages: HTML_QuickForm from 3.2.7 to 3.2.11; HTML_QuickForm_advmultiselect from 0.5.1 to 1.5.1; HTML_QuickForm_Controller from 1.0.3 to 1.0.10; Pager from 2.4.2 to 2.4.8.
http://code.google.com/p/chamilo/source/detail?r=1d2159e3cc28b9cdd7b3d6732820b75aeed913bf&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11032:f90899e256a7Feature #272 - Revision of a "TODO" about PEAR's date control.
http://code.google.com/p/chamilo/source/detail?r=f90899e256a7f321a3a4d28b03442961628b981c&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11066:2965446b937b Feature #272 - Revision and cleaning of the SortableTable class.
http://code.google.com/p/chamilo/source/detail?r=2965446b937b23749ea7244dc988d18350527d49&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11072:0d47b668d273 Feature #272 - Changes that lead loading the file sortabletable.class.php (and the tree of related files) only when it is necessary, on explicit statement require_once. Previously this file loaded on every page, which was waste of computing resource.
http://code.google.com/p/chamilo/source/detail?r=0d47b668d2733308d7f6404a481dd2ac8f99a199&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11073:b9f213ba7a5b Feature #272 - FormValidator.class.php: Revision and cleaning (minor changes).
http://code.google.com/p/chamilo/source/detail?r=b9f213ba7a5bbe6259eff2e36c8bc137f2f337b7&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11075:d98c7faf1535Feature #272 - Revision of Display class. The deprecated metod Display::user_name() has been removed (use api_get_person_name() instead).
http://code.google.com/p/chamilo/source/detail?r=d98c7faf15353e95e7a748099f97e3f300a7f773&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11090:992032232fd0 Feature #272 - Course homepage, making it lighter: Removing unnecessary loading of an old library for debugging purposes.
http://code.google.com/p/chamilo/source/detail?r=992032232fd0617a525b0c9f840eefb8fef470b5&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11106:951826fed7d7 Database::num_rows() method - Adding a validation check for the input parameter. Some corrections in the comments.
http://code.google.com/p/chamilo/source/detail?r=951826fed7d711593be3a3e6921aab3b2e6be624&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11108:0f63f51bfd18 Feature #272 - A minor correction for the SorttableTable class, the distributed with the system PEAR packages are to be loaded for sure.
http://code.google.com/p/chamilo/source/detail?r=0f63f51bfd1869ee825c9baa763b3ab41c654f97&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11165:b5513c124277 Feature #272 - Gradebook: Adding internatioalization-related metadata to a generated html-page.
http://code.google.com/p/chamilo/source/detail?r=b5513c1242778d8694da56ce91afb8a636a7188d&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11169:3d7226ed8ab5 Tasks #272 and #605 - Intrnationalization library: The function get_lang() has been rewoked. Suggestion links to the former language translation tool are not shown anymore. Code has been simplified. The current tool "Chamilo Translation Application" produces UTF-8 strings, so dynamic string purification is no more needed.
http://code.google.com/p/chamilo/source/detail?r=3d7226ed8ab5c8137cf0306c4dc6762f04cdeb9e&repo=classic
Update 21-MAR-2010: Something like explanation for this transaction may be found in my post here: http://www.chamilo.org/en/node/145

11168:bf5fab6db584 Updating language files form Chamilo Translation Application.
http://code.google.com/p/chamilo/source/detail?r=bf5fab6db584414e8afedca505f6c081ed82baee&repo=classic

11167:07108038f282 Feature #272 - text.lib.php: Adding two new functions api_camel_case_to_underscore() and api_underscore_to_camel_case() and tests for them.
http://code.google.com/p/chamilo/source/detail?r=07108038f28239425e9183398d9fb079a16ac79b&repo=classic

11166:9ee94746ca1c Feature #272 - Correcting some tests for the internationalization library.
http://code.google.com/p/chamilo/source/detail?r=9ee94746ca1c38aaaf110cc519a25ec3675b1d08&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11174:1d127fdee9bf Feature #272 - Reworks in the course homepage and the course navigation menu about translation of the tool icons.
http://code.google.com/p/chamilo/source/detail?r=1d127fdee9bfdb3b5032c035b7a85f1b01753a82&repo=classic

11173:748639425a1c Feature #272 - Internationalization library: Additional reworks and corrections about the functions get_lang() and api_is_translated().
http://code.google.com/p/chamilo/source/detail?r=748639425a1c56783eb328f6cbd5e073d1c14d1d&repo=classic

11172:cee0ef875cfe Feature #272 - Course homepage: Some minor optimizations.
http://code.google.com/p/chamilo/source/detail?r=cee0ef875cfe8a711b1dfeb08a0e13db79c88a04&repo=classic

11171:5294f646ac95 Feature #272 - Internationalization lubrary: A minor correction.
http://code.google.com/p/chamilo/source/detail?r=5294f646ac95dc6c6199f90963d6cc327e5de82c&repo=classic

11170:4fe732fdb22c Feature #272 - Internationalization lubrary: Adding a new function api_is_translated().
http://code.google.com/p/chamilo/source/detail?r=4fe732fdb22ce70093c6c4058feaa08f60e8c2ad&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11180:2adc83d37e33 Feature #272 - Help pages: Minor rework and code cleaning.
http://code.google.com/p/chamilo/source/detail?r=2adc83d37e33b7b65af03c368666fe7365df622b&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11230:60df1544c56f Feature #272 - "Documents" tool: Revision again, some cleaning and unrelated changes have been done.
http://code.google.com/p/chamilo/source/detail?r=60df1544c56ffd11dc464fa378a7ad2f0a80c4eb&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11260:20798b6a1f54 Feature #272 - "Surveys" tool: Revision and cleaning, part 1.
http://code.google.com/p/chamilo/source/detail?r=20798b6a1f545ca3e03a83e9de80babd89164801&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11262:429bd924fc51 Feature #272 - "Surveys" tool: Revision and cleaning, part 2.
http://code.google.com/p/chamilo/source/detail?r=429bd924fc51f71533199275817a58cb05146bda&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11281:e63951a62384 Feature #272 - Internationalization library: Correction in the function api_fgetcsv(), removing "\r" or "\n" characters at the end of the processed line.
http://code.google.com/p/chamilo/source/detail?r=e63951a62384dcb7c07c059234ae8d82969b3bfd&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11283:04cad954b82b Feature #272 - import_backup.php: Revision and cleaning, a visual correction has been added.
http://code.google.com/p/chamilo/source/detail?r=04cad954b82bfec885cf4b32826e47ec68b27412&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11284:22cbfb3faa3e Feature #272 - Internationalization library: A change for correct detection of "broken" UTF-8 texts as UTF-8.
http://code.google.com/p/chamilo/source/detail?r=22cbfb3faa3ed5222ca18de9ff021f49ed9884a1&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11285:d532b37e4ffd Feature #272 - create_backup.php: Revision and cleaning, a visual correction has been added.
http://code.google.com/p/chamilo/source/detail?r=d532b37e4ffdd05f2fc67c8294a5dcab2a6747b2&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11287:cf6aabc641e1 Feature #272 - backup.php, copy_course_session.php, copy_course.php, recycle_course.php: Revision and cleaning.
http://code.google.com/p/chamilo/source/detail?r=cf6aabc641e184d80f4dc10d91b180639f8384c3&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11297:ab2f3ef10345 Feature #272 - Reverting the previous detection of "broken" UTF-8, it does not work well, reworked code should be tried.
http://code.google.com/p/chamilo/source/detail?r=ab2f3ef10345540ead0dc01fef2d057185b30929&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11299:17b2949976f4 Feature #272 - Encoding detection improvement for old course archives (for which encoding was not recorded).
http://code.google.com/p/chamilo/source/detail?r=17b2949976f41f1d7513e36e57767b6f15a2a1e1&repo=classic

11298:e191c3680e1e Feature #272 - api_detect_encoding() has been reworked for successfull detection of "broken" UTF-8 texts as UTF-8. A test for this case has been added.
http://code.google.com/p/chamilo/source/detail?r=e191c3680e1ee788485d20036d449282e16ba17b&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11371:d38bbf94b4fc Feature #272 - TableSort class: Cleaning and coding conventions.
http://code.google.com/p/chamilo/source/detail?r=d38bbf94b4fcff8f0ff5079288a9f19703a545df&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11372:7eea5799c2a0 Feature #272 - TableSort class: Some minor optimizations.
http://code.google.com/p/chamilo/source/detail?r=7eea5799c2a06ac1ab1bc8fcc7adbbe1f14b75eb&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11374:d69efd766a79 Feature #272 - TableSort class: Removing old hacks, cleaning and optimization.
http://code.google.com/p/chamilo/source/detail?r=d69efd766a7999497d27a45eb47ed5c5ed16b2fe&repo=classic

11373:88a656db1fad Feature #272 - TableSort class: Adding a new custom method for sorting.
http://code.google.com/p/chamilo/source/detail?r=88a656db1fadf9f42b41439b034f894f49cc78ec&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11431:ec396ee8e22f Task #272: Trying to fix a reported in a private message Fatal error: Class 'IntlDateFormatter' not found in C:\xampp\htdocs\chamilo\main\inc\lib\internationalization.lib.php on line 570
http://code.google.com/p/chamilo/source/detail?r=ec396ee8e22f32c707ab07396c764436b022921b&repo=classic

Update 06-APR-2010: The problem has been reported as solved.

Updated by Ivan Tcholakov over 9 years ago

It seems to me that the internationalization library will "live" longer than I expected. :-) Development of PHP is in a radically new situation as I can estimate. The team is going to choose another way for Unicode?
Future of PHP 6 - http://schlueters.de/blog/archives/128-Future-of-PHP-6.html
Resetting PHP 6 - http://lwn.net/Articles/379909/
Why I am pissed - http://pooteeweet.org/blog/1689
[Joke! I would say a bitter joke] PHP 6 finally released - http://giorgiosironi.blogspot.com/2010/04/php-6-finally-released.html

Updated by Ivan Tcholakov over 9 years ago

11497:6c729c80ae29 Task #272: Internationalization library: Removing a comment about PHP6 that maybe is not actual anymore.
http://code.google.com/p/chamilo/source/detail?r=6c729c80ae2952e903bafb5cc4e2c0f13c990691&repo=classic

Updated by Ivan Tcholakov over 9 years ago

Here are the issues for solving before closing this task:
1. Enlargement of some text database fields as it is described in http://support.chamilo.org/issues/272#note-104
2. A final, informed decision about http://code.google.com/p/chamilo/source/detail?r=88e8119370d7b626582b3fca9b3de33e81532327&repo=classic and making a update in changelog.html, this is needed at the moment. I am neutral about this transaction, I can accept it.
3. Final check for changing some default values from ISO-8859-15 to UTF-8.
4. There is an option "storing web-session in database". To be checked whether table/tables for supporting this option contain binary data only or they contain text fields, i.e. to be checked whether encoding corrections are needed.

Updated by Ivan Tcholakov over 9 years ago

11508:7d750537eeb4 Feature #272 - Lengts of some database text-type fields are increased. The need was identified by using a real course in Bulgarian language.
http://code.google.com/p/chamilo/source/detail?r=7d750537eeb46d80abe611dedbd2ec0ade42c59d&repo=classic

Updated by Ivan Tcholakov over 9 years ago

11509:f8a14c2a0c72 Feature #272: Internationalization library: Some internal default values have been changed from ISO-8859-15 to UTF-8.
http://code.google.com/p/chamilo/source/detail?r=f8a14c2a0c72347511372ce604f6eab4099dce26&repo=classic

Updated by Yannick Warnier over 9 years ago

Ivan Tcholakov wrote:

It seems to me that the internationalization library will "live" longer than I expected. :-) Development of PHP is in a radically new situation as I can estimate. The team is going to choose another way for Unicode?
Future of PHP 6 - http://schlueters.de/blog/archives/128-Future-of-PHP-6.html
Resetting PHP 6 - http://lwn.net/Articles/379909/
Why I am pissed - http://pooteeweet.org/blog/1689
[Joke! I would say a bitter joke] PHP 6 finally released - http://giorgiosironi.blogspot.com/2010/04/php-6-finally-released.html

Yes. We'll see how it all goes, but basically a bunch of big companies should back the PHP development a little bit to help put the necessary work into PHP6, otherwise we'll never see it (the works at hand are too big for a few voluntary people to undertake, I believe).

Updated by Ivan Tcholakov over 9 years ago

Logging some transactions from Yannick:

11512:5311c8489f07 ywarnier 2010-04-10 14:23 -0500 Make sure all fields are equally defined in all courses (part of the process is normally done through the CONVERT TO CHARACTER SET operation, but this might lead to different field sizes in similar tables from different databases or on different systems
http://code.google.com/p/chamilo/source/detail?r=5311c8489f07086fade2682a0fbadd765f5a9810&repo=classic

11511:b344920f5e69 ywarnier 2010-04-10 14:21 -0500 Changed type definition for tag.tag to char() to improve efficiency
http://code.google.com/p/chamilo/source/detail?r=b344920f5e69da1ec7dcf696c5f7c2faafc83204&repo=classic

11510:c1ba8e03b54b ywarnier 2010-04-10 14:20 -0500 Fixed little difference in course creation from previous migration changes in same version
http://code.google.com/p/chamilo/source/detail?r=c1ba8e03b54bbeffa2b0f3920415c691cbeb1fb0&repo=classic

Updated by Yannick Warnier over 9 years ago

Ivan Tcholakov wrote:

2. A final, informed decision about http://code.google.com/p/chamilo/source/detail?r=88e8119370d7b626582b3fca9b3de33e81532327&repo=classic and making a update in changelog.html, this is needed at the moment. I am neutral about this transaction, I can accept it.

Yes, I agree. I know it's dangerous and needs to be tested, but we can't continue supporting two characters sets at the same time in the current structure of chamilo providers (we need to ensure many more providers are trained enough to know how to deal with two character sets).

Updated by Ivan Tcholakov over 9 years ago

About the last issue to be solved:
"4. There is an option "storing web-session in database". To be checked whether table/tables for supporting this option contain binary data only or they contain text fields, i.e. to be checked whether encoding corrections are needed."

This is the case with the following configuration option: $storeSessionInDb = true;

I found that corrections about the encoding of the table chamilo_main.php_session is needed. It should be UTF-8 but connection to is should work with the platform character set, i.e. in the same way as the other tables are accessed. I am preparing the necessary changes.

Updated by Ivan Tcholakov over 9 years ago

11521:0a810085ce6d Feature #272 - Speeding up the function api_get_system_encoding().
http://code.google.com/p/chamilo/source/detail?r=0a810085ce6dd645e1ec495b63c6933b76f556e2&repo=classic

11520:618ad63effe4 Feature #272 - session_handler class: Corrections about the encoding of the database connection used for storing php-session data.
http://code.google.com/p/chamilo/source/detail?r=618ad63effe49ee75164b657e955834ac3376aaa&repo=classic

11519:513c15f1b0ca Feature #272 - session_handler class: Some variables have been reanmed (coding conventions).
http://code.google.com/p/chamilo/source/detail?r=513c15f1b0ca1e3e0549e8504de68b0bb9b8ca20&repo=classic

11518:7878d2a95a62 Feature #272 - Cosmetic changes in the code for session_handler class.
http://code.google.com/p/chamilo/source/detail?r=7878d2a95a62743243f1186af8f0ba4e522ee458&repo=classic

Updated by Ivan Tcholakov over 9 years ago

This is it about #272, I hope. I am going to close this task when I see the next alpha version released.

Updated by Ivan Tcholakov over 9 years ago

A reminder: If you don't need the file chamilo/main/install/dokeos_main.sql anymore, you may remove it.

Updated by Ivan Tcholakov over 9 years ago

11542:31c3a27ce1a5 Task #272 - Adding html-metadata (encoding, language, text direction) in the file lp-save.php.
http://code.google.com/p/chamilo/source/detail?r=31c3a27ce1a5b27084a7224ab621b6eecbd997f3&repo=classic

Updated by Ivan Tcholakov over 9 years ago

  • Status changed from New to Feature implemented
  • % Done changed from 90 to 100

I think, this task has been done. I am closing it.

Also available in: Atom PDF