WebAudioFundamentals.md

Web Audio Fundamentals

HTML 5

Until recently, web developers had to rely on cross-browser plugins (e.g., Flash) to serve and play back sounds on a web site. This finally changed when all of the major browser manufacturers rallyed around the audio tag, introduced in HTML 5. Its key capabilities include: auto playback, preloading sounds, a controls UI, looping, and a src URL for the audio stream. Unfortunately support for audio codecs varies significantly with each browser. Thus, if you're sound file uses the patent encumbered MP3 or AAC, then you will have to provide an alternate format such as the WAV file format, which is lossless and raw. Its typically a much bigger file, but it is supported by all browsers. Below is an simple example of using the audio tag with attributes controls (the standard HTML controls for audio on a web page), autoplay (audio plays automatically) and loop (auto repeat automatically):

<audio src=“audio.ogg" autoplay controls loop>
<p>Your browser does not support the <code>audio</code> element.</p>
</audio>

Caution! - support for Audio Codecs (e.g., MP3) varies with each browser. So if you're looking to deploy to multiple browsers then you will likely need to include multiple file formats for the same audio source.

For most applications, the audio tag will be the only requirement for adding audio to a web site. However it does have its limitations, including:

• No precise timing controls
• A limited number of sounds
• No way to reliably pre-buffer a sound
• No ability to apply real-time effects
• No way to analyze sounds

Consequently, if you are looking to develop a game web site or perhaps provide some mixing, processing, and filtering tasks found in digital audio desktop applications, then you are likely to turn to using the more capable, but significantly more complex Web Audio API.

Web Audio API

Web Audio, proposed by the W3C Audio committee, is a high level JavaScript library for processing and synthesizing audio on the web. The key paradigm is the "Audio Context", borrowed from Apple's Core Audio Library and is the basis for modular synthesizers (e.g., Moog). You can think of the Audio Context as directed graph of nodes. The most simple graph consists of two nodes - a Source Node for your audio source (e.g., MP3 file) and a Destination Node for your output (e.g., computer speakers). Web Audio node types include:

• Source Nodes (sound sources such as audio buffers, live audio inputs and oscillators)
• Modification Nodes (filters, convolvers, panners, etc.)
• Analysis Nodes (analyzers (e.g., frequency, bpm, etc.)
• Destination Nodes (audio outputs and destination buffers)

Initializing, Loading, & Playing Audio

Initializing the Audio Context Graph is shown below.

var context = new AudioContext();

On the front end, set up your pads, knobs, and controls with a unique id attribute, then use JQuery or another framework on the back end to listen for any changes (e.g., user hits play). Below is an example of a Drum Pad with a Volume Control

<div data-pad="pad1">
    <h2>Controls</h2>
    <label for "volume 1">Volume</label> 
    <input type="range" min="0" max="5" 
           step="0.1" value="1" 
           data-control="gain" 
           name="volume1">
</div>

Document Ready

On the back end, after your HTML document has loaded, simply buffer all of your sound files and assign event listeners. Below is an example using JQuery:

$(function() {
    $('#sp div').each(function() {
        addAudioProperties(this);
    });

    $('#sp div').click(function() {
        this.play();
    });

 pad = $(‘#’ + ‘pad1’);

    $(this).data(‘control’) {
        pad.volume.gain.value = 
           $(this).val();
   }
})();

Playing Audio

To associate a pad or button element with the audio source, our anonomous Document Ready function calls the addAudioProperties function, which accepts an element object and adds three properties to this object. The first property is the name which corresponds to the div id. The second property is the URL, which is referenced by a HTML data-sound attribute. The third property is the audio buffer in binary format, containing the audio returned by loadAudio (see below). Finally, the fourth property is added when the element is clicked (i.e., played).

function addAudioProperties(object) {
    object.name = object.id;
    object.source = $(object).data('sound');
    loadAudio(object, object.source);
    object.volume = context.createGain();
    object.play = function () {
        var s = context.createBufferSource();
        s.buffer = object.buffer;
        s.connect(object.volume);
        object.volume.connect(context.destination);
        s.start(0);
        object.s = s;
    }
}

function loadAudio( object, url) {
 
    var request = new XMLHttpRequest();
    request.open('GET', url, true);
    request.responseType = 'arraybuffer';
 
    request.onload = function() {
        context.decodeAudioData(request.response,
        function(buffer) {
           object.buffer = buffer;
        });
    }
    request.send();
}

Suggested Resources

• “Using HTML5 Audio and Video”.,Mozilla Developer Network
• Smus, Boris. Web Audio API. O’Reilly Media, 2013.
• “HTML5 Web Audio API Tutorial: Building a Virtual Synth Pad”, Armando Roggio, Site Point
• “Web Audio API", W3C Working Draft 10 October 2013