Chamilo LMS v10 (or 1.10) developer guide

Chamilo LMS v10 introduces a lot of new stuff and a new structure that is filled with good stuff for the future. This might be a little overwhelming at first, so here is an attempt at explaining you all this in an orderly manner.

Composer

Because there are so many people out there doing great PHP code, and because we can't do it all by ourselves, we decided to board the ship of PHP innovation and the new breed of packaging that was born in 2012: Composer.

It is important to understand Composer, as it will greatly lower your learning curve as you start analysing Chamilo's code.
Composer is not only a packaging system: it is, but it is also a dependency management system, delivered with a little downloadable script that lets you download all the dependencies.
So in an idea world, you would declare a small "Chamilo" package, add a few, high-level, dependencies, then just ask Composer to download the dependencies and... depending on the rules you gave inside the Composer package configuration file, it would download them in the appropriate versions.
You would then only have to install Chamilo and... that's it. In some measure, it's very similar to the Debian packaging system. Only in PHP.

Now the thing with all packaging systems is to have a central repository to store all the packages, otherwise it becomes a CentOS-like nightmare, where you have to rely on a whole bunch of unofficial repositories which all store different versions of the same software. That central repository would be https://packagist.org/. You can navigate it and see what we mean... a whole bunch of structured software, ready to be included in yours.

Symfony2 and Silex

While we started thinking about using Symfony2 as a base for the development of Chamilo LMS, we quickly decided to digress a little: what we wanted from Symfony was mainly its structure, its friendly URLs, and its general handling of HTTP requests. But we didn't particularly want the whole of Symfony.
Adopting the whole Symfony would have meant we would have had to change the whole Chamilo code (re-write most of it) just to be able to publish v10, follow the Symfony2 directory structure, config files, etc and we know some of you wouldn't have had the patience to wait for that (we didn't either).

So Julio went on a Symfony2 training. Then he found "Silex" which is a PHP micro-framework based in Symfony2 components. A mini-router that you can "drop-in" and just use progressively as you move the code from an anarchical structure (not so anarchical, but you must admit that having each tool behave in a different way is a bit difficult to grasp) to a more structured, orderly code.

So, the first reaction you'll have when you start using Chamilo v10 is "why is he sending me to web/index?". Well... that's because this part of Chamilo now goes through Silex first, which manages a whole lot of initialization work, and then passes the handle to whatever script you really wanted to get to, but by the time it gets there, you've been checked for authentication, permissions, routes, etc, and the final script can focus on the essential work.

So no, we don't use the full Symfony (although we use several of its components) but we started integrating a lot of things through Silex, and we think that's for the best.

Coding conventions: now PSR-0 to -3

We've always been using Coding conventions in Chamilo LMS, but recently, with the integration of some Composer components, and the integration of some bits of Chamilo code to composer, plus the writing of more structured unit tests, we've realized that we needed more, and that what we relied on (PEAR2 conventions) was not so much of a conscensus.

So, seeing that Composer suggests we use PSR, a set of internationally-established coding conventions for PHP, and that they weren't too far away from what we were already using... we decided to go for it!

From now on, if you write code for Chamilo LMS, you are expected to write it following PSR-0, PSR-1 and PSR-2. PSR-3 is a little overboard. You can find all these here: https://github.com/php-fig/fig-standards/tree/master/accepted

Single database structure, common DB layer, InnoDB, etc

We're not particularly fans of InnoDB (for one because it's MySQL-specific), but as requirements are getting higher for Chamilo LMS (several portals getting over the 100,000 users limits, one even getting to 600,000), we have had to take measures!

Our first move, when passing from 1.8.* to 1.9.* series, was to unify the database. Now the thing is... we don't really hate spreading the load between several databases (in fact, Michael Widenius, the creator of MySQL, commented that in some cases you might want to do it if you split the databases between several physical servers), but really, the structure we had in 1.8 was inherited from a big mistake made in 2001 by the Claroline team (our oldest ancestor software), and it just bugged every administrator who had to deal with more than 8,000 courses.

So cleaning up the database and allowing it to stand out as a single database with no more than 250 tables, no matter the number of courses, really helped out for many of the big-time Chamilo users. If you're one of these cases of users with a very old, very heavy Chamilo system, please consider migrating at least to 1.9! It might well save your ass.

This first move (unifying databases) was done without much budget, though, and the result is that the migration was a little incomplete, in the sense that unifying 800 tables (in case you had 800 courses) into one requires that you identify each item in that table with a new unique identifier. Sadly, we missed that opportunity and ended up using a combination of c_id (the course, numerical, ID) and id (the previous unique ID for each table) as a primary key.
Now this isn't so bad, as it guarantees unicity of the key, but there are a series of disadvantages using this method:
  • it's a composed key, so building it requires more effort
  • it's a composed key, so indexing it requires more effort
  • InnoDB tables do not allow tables with a composed primary key

Now the fact is that, for huge tables that need updating, MyISAM is quite inefficient, in that it locks the whole table once an update query comes in, and only unlocks the table once it's done. As you can imagine, this might be alright with small tables with, say, a few thousand records. But when you have a few thousand active students, they usually generate a few million records, and they tend to generate update queries much more often. As these lock the tables, all others are waiting and queues start to form.

This (the locking of whole tables) obviously slows down the whole system, so we needed to avoid that as much as possible. This is why, in time for v10, we started the cleaning of the code and the integration of an "iid" field (see #6500) in each previous-course-table, and the corresponding:
  • deletion of the previous, composed, primary key
  • the addition of a unique key constraint on the c_id + id combination (to maintain the unicity of each resource in a course, for now)
  • the change of AUTO_INCREMENT from id to iid
  • the assignation of the primary key to iid

With all this, you get v10 with all tables in InnoDB by default (although you can still change that to other engines if you want) and that results in a much faster database and the possibility (between others) to implement truly-redundant database clusters like Percona or Galera. This is great for optimization and very-large-scale portals. If you don't understand it now and you just started your Chamilo portal, remember this... one day, you'll thank us for taking such a proactive approach :-)

Database layer (Doctrine2)

Because the ext/mysql PHP extension will be deprecated since PHP 5.5 we needed to upgrade our "Database" class. We did this using Doctrine DBAL in combination with the Doctrine Service Provider.

So this is working for MySQL databases now. We didn't test it with other drivers (Oracle, PostgreSQL, etc) but with each version passing you could run Chamilo LMS on an Oracle database (but why would you want to do that with PostgreSQL around, really?).

New file structure

Because we wanted Chamilo LMS v10 to be much more widely available to mankind, we decided to force our way to the "packageable" state, where our directories are ordered in a way that allows most UNIX-based operating systems to know exactly what permissions have to be applied to what directories.
This is why you'll find Chamilo LMS v10 has a considerably updated file structure. Something like this:
  • config/ will contain all configuration files. These should be written at install time, then set as read-only
  • data/ will contain all the data generated through Chamilo for course and users material
  • documentation/ contains statis documents about Chamilo, for documentation purposes
  • logs/ contains data registered during specific events in Chamilo
  • main/ contains the legacy (still in use!) code of Chamilo - new code is generally added in src/
  • plugin/ contains the plugins that one may add to Chamilo - we are considering moving this inside data
  • src/ contains the new code developed for Chamilo
  • temp/ contains temporary files created for export purposes - should be cleaned regularly through a cron script
  • tests/ (only for people downloading the source from Git)
  • vendor/ where all the Composer packages are stored (including Chash, which includes the install scripts)
  • web/ fake directory for redirections to Silex

Debugging

With this version, we added some additional, centralized, debugging features. If you want to debug Chamilo, get to the end of your config/configuration.php file and change the following to true:

$_configuration['debug']             = false;

Profiling

Some of the Symfony elements are really great for developers, and so is the Symfony Profiler! If you want to know what's going on (this will also help for debugging) during the PHP processing of your script, go to your config/configuration.php file and change the following to true:

// Show a useful toolbar with memory used, loaded time, request, session, logs information.
$_configuration['show_profiler']     = false;

This setting will slow down your portal considerably, so please, use only in development environments!

Critical options moved to configuration.php

Given the fact that more and more people are needed to manage increasingly big portals, it happens more frequently that untrained staff are requested to act as administrators in Chamilo. We encourage discovery of our platform, of course, but sometimes you just don't want to risk some data getting lost or your portal getting unusable because somebody tried a little too much... (see what we mean?)

This is the reason why we are moving some of the most dangerous or misunderstood settings of Chamilo to the configuration file: out of reach of those beginner admins who might just click the wrong button (and then confirm the wrong popup, and still do one more mistake before calling you...).

You will now find the following options (some of them are new) only in the config/configuration.php file:

$_configuration['security_key'] = '60e2264a571c973fcbf5af12b69de595'; // Was already in configuration.php, but now no part of it is used in the e-mail sending anymore (it was a security risk)
$_configuration['deny_delete_users'] = false; //Deny the removal of users
$_configuration['login_as_forbidden_globally'] = false; //Prevent all admins from using the "login_as" feature.
$_configuration['debug']             = false; //previously the "Test mode server" option 
$_configuration['platform_charset']  = 'utf-8'; //previously the "Platform charset" option in the languages section

Chash

For the developer and admins who like the command line, you are going to love this script!
Chash (the "Chamilo Shell") is inspired from Drush (the "Drupal Shell") and allows you to do a series of very complex operations in just one or two commands.
Read more about Chash on its Github page: https://github.com/chamilo/chash

CSS and themes

It was already there in 1.9.*, but we didn't really publicise it yet: in Chamilo LMS v10, you can do CSS styling, like you used to do in 1.9, but you can also do templating (or theming if you come from the Drupal world).

Theming with Twig

Starting a new theme in v10 is easy: go to the main/template/ and copy the "extendme" theme. This one "extends" (in reality it just uses) the "default" theme.
This means that you can start with that and have a fully-functional separate theme to play with. On your development platform, switch the active theme by going to the Admin page -> Platform settings -> Stylesheets -> there you'll find a "Template" box. Just write the name of your new template directory (e.g. "mykillertemplate") and save. That's it, you're using your theme now!

Because theming can present a lot of issues, we recommend you always enable debug and profiling in your configuration file (see a few sections above for that). This will explain much more clearly when, for example, there is a syntax error or a file not found error in one of your .tpl.

We use the Twig templating engine, which is really similar to Smarty, if you're used to that. Checking the existing definitions in the "default" theme, you shouldn't have any problem finding what you need.
You will probably want to scan through main/inc/lib/template.lib.php, main/inc/lib/page.lib.php and main/inc/lib/display.lib.php before you embark on the templating cruise: it will help you out knowing what's available to you.

For example, if you want to move the menu block on the right side like it was in Chamilo 1.8, you should edit the layout/layout_2_col.tpl file (take some inspiration by looking at the main/template/default/layout/layout_2_col.tpl default tpl file).

If you ever get a rare issue that looks like it's not taking your latest changes into account, just delete the contents of the /temp/twig/ directory, and you'll be up and running again.

CSSing

Changing CSS is very easy, just copy one of the subdirectories in main/css/, go to your Admin -> Platform settings -> Stylesheets page, select it, and start hacking! Together with Firebug, you're in for a lot of fun.