What is it ?
WebSID is an in-browser synthesizer that approximates (purists might argue it's a poor "emulation" of) the sound of the Commodore 64, the popular home computer of the 1980's whose SID chip (either MOS 6581 or later MOS 8580 chips) defined much of what we now know as the "characteristic sound of retro videogames".
The chip had its fair share of limitations : you could only play three channels simultaneously, which led to the introduction of playing arpeggios (the fast, rippling sequence of notes) instead of multi-timbral chords, just so the remaining two channels would remain free to synthesize other sounds (e.g. to keep a bass-line and drum beat going).
You can jump straight to WebSID and start playing chiptune right now!
How did it come about ?
WebSID was initially created as a Chrome experiment to test-drive the possibilities of synthesizing audio inside a web browser in real-time using the experimental Web Audio API.
After the initial release gained popularity (after both Google as well as the audio community embraced the application), WebSID has been updated regularly to meet feature requests and introduce new functionality. WebSID currently features the following:
- Record, playback, save and share songs with friends
- Fully responsive, works on iOS and Android tablets and phones
- Connect MIDI keyboards to play WebSID using a keyboard (Chrome only)
- Works offline
- Fully threaded audio rendering
Web Audio... how does that work out ?
First up, Web Audio has nothing to do with the "HTML5" <audio>-tag. Web Audio provides an API to generate and manipulate audio in real time, all within the web browser. The API provides a plethora of modules out of the box, such as filters, oscillators, wave tables and even convolution reverb.
Though the API's modules are accessible in JavaScript, they actually run outside of the VM on the native layer of the CPU, providing high performing results. You can elegantly chain these "nodes" together to quickly sketch an aural idea.
While it is recommended to create your custom audio processors by combining these supplied modules, the
ScriptProcessorNode AudioWorkletNode is likely to be of most interest to those who wish to synthesize and
generate their audio in real time using their own custom synthesis routines.
So how does this work ?
On audio processing
Audio is processed one buffer at a time in a ring buffer queue (consisting of two buffers, where one is currently playing back its contents, while the other is queued for playback, a familiar concept in digital signal processing).
Each time a new buffer presents itself for enqueue-ing, a processing callback is fired on the AudioNode. At this point the buffer is available to JavaScript and you can use JavaScript and all the math you can muster to fill the outputBuffer with audio.
When WebSID was first released, this process was handled through the now deprecated ScriptProcessorNode. This has since been updated to use the AudioWorkletNode instead, though the principle remains the same.
Let me use this space to state that when WebSID was created, JavaScript felt utterly useless at calculating complex algorithms in real time like those used in signal processing and synthesis. The premise of running code in a VM, "aided" by garbage collection in a single thread makes this task very hard indeed, especially if there is also a UI to render.
Update: with the advent of AudioWorkletNode replacing ScriptProcessorNode, a true threaded environment for audio rendering is realized. Also WASM is now a thing.
Not all was bleak, as JavaScript also provides an elegant base for creating applications. So with paying a little attention, a lot of mileage can be gained just by knowing how to deal with the challenges, which brings us to:
WebSID "under the hood"
Sequencing and timing: the audioContext has the most stable and precise clock available to JavaScript, using time (in seconds) as a floating point value it is possible to enqueue events at a specific start time (relative to when the audioContext was created), enabling us to enqueue and play back events in a musical composition with the highest precision.
When playing one of WebSIDs keyboards, the notes are generated as a value object (VO) representing properties like frequency (the pitch of the note in Hz) and phase (used to generate the waveform). This object is added to an "event queue" which basically holds all the notes that should be synthesized into the enqueued output buffer.
When playing back a pre-recorded composition using the sequencer, the notes are enqueued by using temporary OscillatorNodes. These start their oscillation (silently!) at the currentTime of the audioContext and stop at the time the note should start playing, firing an ended-Event.
Upon receiving the ended-callback we instantly create a VO for the note and add it to the event queue to be synthesized instantly. Basically, the OscillatorNodes are used solely as a highly accurate clock.
Sequencing the sequencer
At a regular interval, the sequencer checks which events (inside the currently playing/enqueued measure) should start their playback for the next timeframe.
E.g.: every 25 milliseconds the sequencer checks the events that should start playing within now and the next 100 milliseconds and enqueues them using the "Oscillator clock"-trick mentioned above.
Note the lookahead is larger than the interval to overcome notes dropping out due to drifting interval callbacks.
Optimization techniques
The garbage collector is your enemy. Seriously. Without using workers, this will eventually stall timers, queues, etc. even on fast machines. Even worse, OscillatorNodes that are created and aren't strongly referenced (thus become eligible for garbage collection as the creating functions body exits), will never fire their attached callback handlers after they are cleaned up. Which is ironic, given how most JavaScript memory leaks occur due to dangling listeners!
Apart from keeping strong references, Object pooling is highly beneficial as it overcomes unnecessary allocation and de-allocation of Objects, especially convenient when you need a lot of "temporary" Objects.
Web Workers
JavaScript is single threaded and as such the rendering of the UI might suffer during heavy calculations, as well as the timer accuracy of setInterval/setTimeout. To overcome this, WebSID used two Workers that operate outside of the main execution thread.
Web Workers provided a means to run the ScriptProcessorNodes callbacks in background threads, separated from the main thread that is responsible for rendering the user interface. The Workers operate in a different scope than the main thread with no access to the document, but for our purposes they are very useful (and without having to worry about mutual exclusion!).
One Worker simply handles the 25 ms interval at which the sequencer should update the event queue to enqueue the next series of events for playback. As this occurs outside of the main execution thread, this timer is less prone to "wander" under heavy load or on garbage collection.
The other Worker was invoked when the ScriptProcessorNode fires its onaudioprocess-callback. It receives an Array containing the event queue, and synthesizes its output into a Float32Array which is returned to the main application to be written into the AudioProcessingEvents outputBuffer.
The ScriptProcessorNode worker is now fully replaced with an AudioWorkletNode providing true threading.
WebMIDI
A nice feature currently available only to Chrome users is the ability to connect MIDI peripherals to your computer and read / send MIDI messages from / to them from your web browser.
As the feature is still experimental and it is likely that the draft will change (not to forget whether it will be adopted by most browsers at all) libraries are scarce or outdated.
As such I wrote a small adapter interface called zMIDI to transfer messages from connected musical hardware and make them available through JavaScript.
You can view more details about zMIDI here.
Languages and technologies used:
JavaScript, Web Audio API, Web MIDI, Web Workers, HTML, CSS, PHP, MySQL.
DSP / synthesis-related
Those who want to know what makes a Commodore 64 sound like a Commodore 64, you can easily view the contents of the worklet that renders the audio here. It's hardly a big secret, it's known as pulse width modulation!
VST plugin version
WebSID is also available as a VST plugin for macOS and Windows, which in turn is fully open source.