Feature #7148

Updated by Yannick Warnier over 5 years ago

With the advent of open audio formats and the availability of audio frameworks in HTML5, it is now feasable to analyse the audio frequency of a recorded voice and *compare* it to another recorded voice, or even a live audio stream being pronounced by a user.

This makes it an ideal case for exercises of type "Audio pronounciation".

The concept would be simple:
* the teacher records an audio and puts it into an audio-type question, then sets a minimum "pass" percentage for the question (+ a score received if succeeded)
* the student takes the test
* the question shows as an audio player with a frequency signal below
* while "playing" the sound, a vertical bar moves through the frequency
* the student click "record" and uses his/her microphone through HTML5 + WebAudio API
* while recording, the frequency of the sound appears to the student as another box below the first one
* the student presses the "stop" button
* the differences in frequency are analyzed and a score is given
* if the score is passed, the student frequency appears on green background and the student receives the success score (0/1) when moving to the next question

The algorithm of comparison wouldn't be *that* easy, as we need to include aspects like:
* frequency
* pitch
* speed
* strength
* starting time (recording might start only a certain time after the student pressed "record")

These parameters, considered as a whole, should provide a good overall result of how close the student was from the recording.
Obviously, these settings (thresholds) should be easily configurable by the developers or ideally by the platform administrator and/or the course teacher.

The visual representation doesn't have to be "complete" or "precise". It just needs to be a visual representation of how wrong the student might have been.

h2. Project planning

* Identify audio-analysis JS libraries
* Identify customizable audio-recording and -playing JS libraries
* Implement recorder (HTML5-based, OGG format)
* Implement player of response audio (with debug to console.log) (HTML5-based, OGG format)
* Implement comparison algorithm (with debug to console.log)
* Tune variables of algorithm
* Integrate into exercises flow of Chamilo

h2. Possible sources of inspiration

* Duolingo app (it doesn't show a frequency, but it gets the same result)
* Any advanced desktop-based language-learning application
* The images attached