Feature #7148

New question type: Audio frequency analysis

Added by Yannick Warnier over 7 years ago. Updated over 5 years ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:
150.00 h
Spent time:
SCRUM pts - complexity:


With the advent of open audio formats and the availability of audio frameworks in HTML5, it is now feasable to analyse the audio frequency of a recorded voice and compare it to another recorded voice, or even a live audio stream being pronounced by a user.

This makes it an ideal case for exercises of type "Audio pronounciation".

The concept would be simple:
  • the teacher records an audio and puts it into an audio-type question, then sets a minimum "pass" percentage for the question (+ a score received if succeeded)
  • the student takes the test
  • the question shows as an audio player with a frequency signal below
  • while "playing" the sound, a vertical bar moves through the frequency
  • the student click "record" and uses his/her microphone through HTML5 + WebAudio API
  • while recording, the frequency of the sound appears to the student as another box below the first one
  • the student presses the "stop" button
  • the differences in frequency are analyzed and a score is given
  • if the score is passed, the student frequency appears on green background and the student receives the success score (0/1) when moving to the next question
The algorithm of comparison wouldn't be that easy, as we need to include aspects like:
  • frequency
  • pitch
  • speed
  • strength
  • starting time (recording might start only a certain time after the student pressed "record")

These parameters, considered as a whole, should provide a good overall result of how close the student was from the recording.
Obviously, these settings (thresholds) should be easily configurable by the developers or ideally by the platform administrator and/or the course teacher.

The visual representation doesn't have to be "complete" or "precise". It just needs to be a visual representation of how wrong the student might have been.

Project planning

  • Identify audio-analysis JS libraries
  • Identify customizable audio-recording and -playing JS libraries
  • Implement recorder (HTML5-based, OGG format)
  • Implement player of response audio (with debug to console.log) (HTML5-based, OGG format)
  • Implement comparison algorithm (with debug to console.log)
  • Tune variables of algorithm
  • Integrate into exercises flow of Chamilo

Possible sources of inspiration


frequency-match.jpeg (14.3 KB) frequency-match.jpeg Yannick Warnier, 07/05/2014 19:52
frequency-match2.jpeg (10.8 KB) frequency-match2.jpeg Yannick Warnier, 07/05/2014 19:52



Updated by Yannick Warnier over 7 years ago

  • Description updated (diff)

Updated by Francis Gonzales over 7 years ago

Some days ago I found these examples that can help:


Updated by Yannick Warnier over 7 years ago

Francis Gonzales wrote:

Some days ago I found these examples that can help:

The last one has been forked and advanced further here:


Updated by Yannick Warnier over 7 years ago

A nice tutorial to show live audio frequency analysis:

<!doctype html>
  <style type="text/css">
   div#mp3_player{ width:500px; height:60px; background:#000; padding:5px; margin:50px auto; }
   div#mp3_player > div > audio{ width:500px; background:#000; float:left; }
   div#mp3_player > canvas{ width:500px; height:30px; background:#002D3C; float:left; }
   // Create a new instance of an audio object and adjust some of its properties
   var audio = new Audio();
   audio.src = 'track1.mp3';
   audio.controls = true;
   audio.loop = true;
   audio.autoplay = true;
   // Establish all variables that your Analyser will use
   var canvas, ctx, source, context, analyser, fbc_array, bars, bar_x, bar_width, bar_height;
   // Initialize the MP3 player after the page loads all of its HTML into the window
   window.addEventListener("load", initMp3Player, false);
   function initMp3Player(){
    context = new webkitAudioContext();
    // AudioContext object instance
    analyser = context.createAnalyser();
    // AnalyserNode method
    canvas = document.getElementById('analyser_render');
    ctx = canvas.getContext('2d');
    // Re-route audio playback into the processing graph of the AudioContext
    source = context.createMediaElementSource(audio);
   // frameLooper() animates any style of graphics you wish to the audio frequency
   // Looping at the default frame rate that the browser provides(approx. 60 FPS)
   function frameLooper(){
    fbc_array = new Uint8Array(analyser.frequencyBinCount);
    ctx.clearRect(0, 0, canvas.width, canvas.height);
    // Clear the canvas ctx.fillStyle = '#00CCFF';
    // Color of the bars bars = 100;
    for (var i = 0; i < bars; i++) {
     bar_x = i * 3;
     bar_width = 2;
     bar_height = -(fbc_array[i] / 2);
     //fillRect( x, y, width, height )
     // Explanation of the parameters below
     ctx.fillRect(bar_x, canvas.height, bar_width, bar_height);
  <div id="mp3_player">
   <div id="audio_box"></div>
   <canvas id="analyser_render"></canvas>

Technically, we should see the framework differently here, and divide the frequency canvas into sections that are proportional to the sound length and progressive (moving X axis) with only one bar that shows (mainly a human voice frequency or something like that).
I'm not sure exactly which frequency is shown when it is used to represent only one bar for a whole split-second... I will have to research that.


Updated by Yannick Warnier over 7 years ago

  • Assignee set to anibal copitan

Updated by Yannick Warnier about 7 years ago

  • Assignee changed from anibal copitan to Imanol Losada

Updated by Yannick Warnier over 5 years ago

  • Target version changed from 2.0 to 3.0

Updated by Yannick Warnier over 5 years ago

There's a basic implementation here, but it requires forking and integrating into Chamilo. This work was made for BeezNest but was not finished, so we still need to evaluate its quality.


Updated by Yannick Warnier over 5 years ago

  • Status changed from New to Assigned
  • Assignee changed from Imanol Losada to Angel Quiroz

Updated by Yannick Warnier over 5 years ago

  • Target version changed from 3.0 to 2.0

Also available in: Atom PDF