• the web audio api (WAAPI) only provides block-based nodes, with the exception of the ScriptProcessor node and the AudioWorklet.
• This precludes lots of important synthesis techniques (feedback, hard sync, audio rate modulation of scheduling etc.)
• The ScriptProcessor and the AudioWorklet let you write sample-level processing using JavaScript. How can we write DSP in a dynamic language like JavaScript that is efficient enough to explore such techniques?
• genish.js: A collection of DSP helpers for dealing with phase, buffers, math operations, and control flow. Optimized for efficiency by avoiding branching, closures, and inheritance, and by only using a single block of memory. Loosely based on Gen for Max_MSP_Jitter.
• gibberish.js: Synths, effects, and sequencing using functions created with genish.js.
• gibber.audio.lib: Rapid development and live coding using synths and effects from gibberish.js. Music theory, synth presets, easy sequencing, patterns and pattern transformations.
• genish.js
• Generates optimized callback functions or AudioWorkletProcessor classes.
• Uses a single memory heap for the compiled function. The memory is managed via a small library I wrote: GitHub - charlieroberts/memory-helper: A small utility class for managing blocks within JS TypedArrays
• avoids dynamic lookups in the compiled functions as much as possible (no upvalues, no prototype chains, no global references).
• avoids branching to help out the JIT compiler; however, sometimes branching is inevitable.
• Demos: hard sync, audio-rate modulation.
• Pros:
- pretty fast! Even beats native WAAPI ugens in some cases (although it is much worse in others)
• Cons: - Writing DSP without operator overloading Is lame.
• GitHub - charlieroberts/jsdsp: A small Babel plugin enabling operator overloading with genish.js
• jsdsp is a small plugin for Babel, a JS compiler. It looks for operators that are operating on genish.js ugens and then replaces them with genish.js functions. For example:
// the following jsdsp code...
{
'use jsdsp'
mod = cycle( 2 ) * 20
car = cycle( 440 + mod ) * .1
}
// gets compiled into this:
mod = mul( cycle(2), 20 )
car = mul( cycle( add( 440, mod ), .1 )For longer, complex ugens this becomes a great shorthand and is very useful.
• Gibberish | A JavaScript Library for Musical Synthesis and Scheduling.
• Builds synthesis and effects ugens using genish.js, and then assembles these into optimized callback functions using second layer of code generation. All genish.js ugens in a Gibberish graph will use the same memory heap.
• Enables single-sample feedback loops between unit generators.
• Enables simple audio-rate modulation of scheduling.
• Two types of sequencers:
• Sequencer - Uses a global priority queue that is checked once per-sample, enabling sample-accurate scheduling. This sequencer is not driven by the master callback function; it simply advances by a value of one on each sample.
• Sequencer2 - This sequencer exposes a rate parameter that can be modulated in the main audio callback.
• Master callback: https://github.com/gibber-cc/gibberish/blob/v3_worklet/js/workletProcessor.js#L120
• Demos: FM w/ feedback, modulating scheduling
In prior versions of gibberish, we were regenerating the callback function and recompiling it every time a property of a ugen changed.
• This means that every the frequency of an oscillator, the gain of a synth, or the cutoff frequency of a filter is changed by sequencing or user interaction, than the entire callback would be regenerated.
• I am currently experimenting with an alternative to this that, instead of writing property values directly into callback bodies, provides memory addresses. The callback function does not need to be recompiled when the values in these memory locations change.
• This is more efficient in scenarios when there are lots of changes to ugen properties, for example, a live coding performance. However, it is less efficient for scenes that are relatively static… baking the property values directly into the callback will be more efficient than looking up memory addresses here. One example use case where it might make more sense to use baked in values would be an effects processing box, where fx parameters are only changed based on user interaction.
• Another option would be to redo codegen whenever a property value is set to not be a number; otherwise values could simply be looked up from the shared memory heap. Haven’t tried this but it might provide the best of both worlds.
Both genish.js and gibberish.js support AudioWorklets, which are intended to eventually replace the ScriptProcessor node for developers who want to write sample-level audio callbacks using JavaScript (or using WebAssembly, which currently powers browser-based versions of Faust and Csound).
The big advantage of AudioWorklets vs ScriptProcessor is that AudioWorklets run in a separate thread from networking / interaction / graphics… ScriptProcessor runs in the main thread, leading to all sorts of pops and glitches if you’re not extremely careful.
The disadvantage is that suddenly we have to worry about inter thread communication. AudioWorklets use an object named the MessagePort to communicate between the main thread and the audio processing thread. A lot of work in recent versions of genish.js / gibberish.js went into ensuring that users don’t have to explicitly think about this. Extensive use of metaprogramming makes code for AudioWorklets look almost identical to code targeting a ScriptProcessor node.
We accomplish this by partially mirroring state between the two threads. Whenever a change to the state of a ugen is made in the main thread, a message is sent through the MessagePort telling the associated audio thread ugen to update. Likewise, whenever changes to the state of ugens occur in the audio thread (due to sequencing), these changes are sent back to the main thread at the end of each block of audio processing.
gibber.audio.lib is a set of abstractions above gibberish.js (which in turn is built on genish.js). It adds a set of presets, music theory, and easy scheduling capabilities to gibberish. For example, the following code in gibberish:
syn = Gibberish.instruments.Synth()
chr = Gibberish.effects.Chorus()
syn.connect( chr )
chr.connect( Gibberish.out )
seq = Gibberish.Sequencer({
target:syn,
key:'note',
values:[55,110,220,330],
timings:[11025]
}).start()… can be reduced to three lines using gibber.audio.lib:
syn = Synth()
syn.fx.add( Chorus() )
syn.note.seq( [55,110,220,330], 1/8 )
This is amazing, I love the work you’ve been doing on Gibberwocky, Gibberish etc.
How does one use AudioWorklets rather than ScriptProcessorNode in gibberish and genish?
And do you feel these two offer things that are not possible when using gen inside Gibberwocky ?
In other words what is the current bleeding edge of these environments in terms of sequencing and dsp live coding, DAW considerations aside ?