Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?

Overview

In order to solve the "data race" issue in WebAudio (bug 22725), this proposal restricts access to the internal PCM data of an AudioBuffer. This does not require AudioBuffers to be immutable, but that modifying their contents occurs through an API which enforces no-data-race semantics.

Since this will involve breaking API changes, this proposal incorporates suggestions from Issue 48 to simplify the AudioBuffer interface, and will also adopt a constructor, as opposed to a factory method.

The new constructor provides an upgrade path for existing pages while not breaking those pages. The factory method can continue to create the "legacy" AudioBuffer in prefixed implementations.

Interface Changes

AudioBufferChannel

This interface represents a single channel of an AudioBuffer.

interface AudioBufferChannel {
    readonly attribute unsigned long length;

	void set(Float32Array array, optional unsigned long offset);
	Float32Array slice(long begin, optional long end);
	void copyTo(Float32Array array, optional unsigned log offset, optional unsigned long begin, optional long end);
};
Algorithms

An AudioBufferChannel is immutable if its associated AudioBuffer is immutable.

Attributes

length

  • Length of the PCM audio data in sample-frames.
Methods

set

  • Copies the Float32Array representing the PCM audio data.

  • An INDEX_SIZE_ERR exception must be thrown if the length of array plus offset exceeds the length of the available audio data.

  • A NO_MODIFICATION_ALLOWED_ERR exception must be thrown if the AudioBufferChannel is immutable.

slice

  • Returns a copy of the underlying PCM audio data. If either begin or end is negative, it refers to an index from the end of the audio data.

  • If end is unspecified, the copy will include all the data from begin to the end of the audio data.

  • The range specified by begin and end is clamped to the valid range for the audio data. If the computed length of the copied data would be negative, it is clamped to zero.

copyTo

  • Copies the underlying PCM audio data to array. If either begin or end is negative, it refers to an index from the end of the audio data.

  • If end is unspecified, the copy will include all the data from begin to the end of the audio data.

  • The range specified by begin and end is clamped to the valid range for the audio data. If the computed length of the copied data would be negative, it is clamped to zero.

  • An INDEX_SIZE_ERR exception must be thrown if the length of array minus offset is less than begin minus end.

AudioBuffer

[
	Constructor(unsigned long numberOfChannels, unsigned long length, float sampleRate),
	Constructor(sequence<Float32Array> channels, float sampleRate)
]
interface AudioBuffer {

    readonly attribute float sampleRate;

    // in seconds 
    readonly attribute double duration;

    readonly attribute AudioBufferChannel[] channels;
};
Algorithms

An AudioBuffer is immutable if it is associated with a live AudioNode.

Constructors

The first constructor creates an AudioBuffer containing a set of empty (zero-filled) AudioBufferChannels of duration length / sampleRate.

The second constructor creates an AudioBuffer by copying the contents of channels into a new set of AudioBufferChannels whose length is the length of the longest Float32Array in channels.

Attributes

sampleRate

  • The sample-rate for the PCM audio data in samples per second.

duration

  • Duration of the PCM audio data in seconds.

channels

  • An array of AudioBufferChannel objects representing the PCM audio data for each audio channel.

  • This array is of fixed length and is read-only.

AudioProcessingEvent

To facilitate transferring output buffers into a ScriptProcessorNode, the readonly keyword on AudioProcessingEvent.outputBuffer is removed.

partial interface AudioProcessingEvent {
	readonly attribute AudioBuffer inputBuffer;
	attribute AudioBuffer outputBuffer;
}

The initial value of inputBuffer is an AudioBuffer with a length equal to the ScriptProcessorNode target's bufferSize, with a channels attribute containing numberOfInputChannels entries, each of which contains audio data generated by the AudioNode connected to the ScriptProcessorNode's input.

The initial value of outputBuffer is an AudioBuffer with a length equal to the ScriptProcessorNode target's bufferSize, with a channels attribute containing numberOfOutputChannels entries, the contents of each of which is empty (zero-filled).

AudioBufferSourceNode

Algorithms

An AudioBufferSourceNode's live state is set immediately after its start() method is called, and its live state is cleared immediately before an ended event is dispatched to it.

Attributes

buffer

  • A NO_MODIFICATION_ALLOWED_ERR exception must be thrown if node is live.

ConvolutionNode

Algorithms

A ConvolutionNode's live state is set immeditely after its connect() method is called, and its live state is cleared immediately after its disconnect() method is called and no additional output connections remain.

ScriptProcessorNode

Algorithms

The inputBuffer and outputBuffer attributes of AudioProcessingEvent are assocated with that event's target ScriptProcessorNode, and that node's live state is set immeditely before the AudioProcessingEvent is dispatch to it, and its live state is cleared immeditately after the event handler for that event completes.

Memory and Performance Considerations

Certain of these modifications to the WebAudio API will necessarily result in increased memory usage for typical operations, while with other operations, memory usage will remain unchanged.

Decoding coded audio

The use case of decoding and playing coded audio data should not be impacted by this proposal.

Playback of generated audio

Playback of generated audio through an AudioBufferSourceNode may require an additional memory allocations and copies, depending on the nature of the generated audio. Specifically, generating audio data into a Float32Array then copying that data into an AudioBuffer will necessarily require an additional copy.

Generating audio directly into an AudioBuffer using the new set() method will still result in 0-additional allocations or copies.

Processing decoded data in ScriptProcessorNode

Writing to output buffers will potentially require a memory copy where previously only a buffer transfer was required. As an output buffer is always provided, this requires no additional memory allocation.

A buffer transfer is still possible by assigning the outputBuffer attribute of AudioProcessingEvent directly.

Readback of AudioBufferChannel data

Reading the internal buffers of AudioBufferChannel objects will now always require memory to be copied. The web author can choose whether to use the AudioBufferChannel.slice() method, which will cause an additional memory allocation, or the Float32Array.set() method, which will copy into pre-allocated memory.

Backwards Compatibility Not normative

WebKitAudioBuffer

interface WebKitAudioBuffer : AudioBuffer {
	readonly attribute length;

    readonly attribute long numberOfChannels;

    Float32Array getChannelData(unsigned long channel);
};

WebKitAudioContext

partial interface WebKitAudioContext {
	WebKitAudioBuffer createBuffer(unsigned long numberOfChannels, unsigned long length, float sampleRate);	
};

WebKit will continue to support WebKitAudioBuffer objects in the prefixed version of the API for every attribute or parameter which currently specifies AudioBuffer. WebKit will drop WebKitAudioBuffer support in the unprefixed API.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.