What is it ?
WebSID is an in-browser synthesizer that approximates (purists might argue it's a poor "emulation" of) the sound of the Commodore 64, the popular home computer of the 1980's whose SID chip (either MOS 6581 or later MOS 8580 chips) defined much of what we now know as the "characteristic sound of retro videogames".
The chip had its fair share of limitations : you could only play three channels simultaneously, which led to the introduction of playing arpeggios (the fast, rippling sequence of notes) instead of multi-timbral chords, just so the remaining two channels would remain free to synthesize other sounds (e.g. to keep a bass-line and drum beat going).
You can jump straight to WebSID and start playing chiptune right now!
How did it come about ?
WebSID was initially created as a Chrome experiment to test-drive the possibilities of synthesizing audio inside a web browser in real-time using the experimental Web Audio API.
After the initial release gained popularity (after both Google as well as the audio community embraced the application), WebSID has been updated regularly to meet feature requests and introduce new functionality. WebSID currently features the following:
- Record, playback, save and share songs with friends
- Fully responsive, works on iOS and Android tablets and phones
- Connect MIDI keyboards to play WebSID using a keyboard (Chrome only)
- Works offline
- Fully threaded audio rendering
Web Audio... how does that work out ?
First up, Web Audio has nothing to do with the "HTML5" <audio>-tag. Web Audio provides an API to generate and manipulate audio in real time, all within the web browser. The API provides a plethora of modules out of the box, such as filters, oscillators, wave tables and even convolution reverb.
While it is recommended to create your custom audio processors by combining these supplied modules, the
ScriptProcessorNode AudioWorkletNode is likely to be of most interest to those who wish to synthesize and
generate their audio in real time using their own custom synthesis routines.
So how does this work ?
On audio processing
Audio is processed one buffer at a time in a ring buffer queue (consisting of two buffers, where one is currently playing back its contents, while the other is queued for playback, a familiar concept in digital signal processing).
When WebSID was first released, this process was handled through the now deprecated ScriptProcessorNode. This has since been updated to use the AudioWorkletNode instead, though the principle remains the same.
Update: with the advent of AudioWorkletNode replacing ScriptProcessorNode, a true threaded environment for audio rendering is realized. Also WASM is now a thing.
WebSID "under the hood"
When playing one of WebSIDs keyboards, the notes are generated as a value object (VO) representing properties like frequency (the pitch of the note in Hz) and phase (used to generate the waveform). This object is added to an "event queue" which basically holds all the notes that should be synthesized into the enqueued output buffer.
When playing back a pre-recorded composition using the sequencer, the notes are enqueued by using temporary OscillatorNodes. These start their oscillation (silently!) at the currentTime of the audioContext and stop at the time the note should start playing, firing an ended-Event.
Upon receiving the ended-callback we instantly create a VO for the note and add it to the event queue to be synthesized instantly. Basically, the OscillatorNodes are used solely as a highly accurate clock.
Sequencing the sequencer
At a regular interval, the sequencer checks which events (inside the currently playing/enqueued measure) should start their playback for the next timeframe.
E.g.: every 25 milliseconds the sequencer checks the events that should start playing within now and the next 100 milliseconds and enqueues them using the "Oscillator clock"-trick mentioned above.
Note the lookahead is larger than the interval to overcome notes dropping out due to drifting interval callbacks.
Apart from keeping strong references, Object pooling is highly beneficial as it overcomes unnecessary allocation and de-allocation of Objects, especially convenient when you need a lot of "temporary" Objects.
Web Workers provided a means to run the ScriptProcessorNodes callbacks in background threads, separated from the main thread that is responsible for rendering the user interface. The Workers operate in a different scope than the main thread with no access to the document, but for our purposes they are very useful (and without having to worry about mutual exclusion!).
One Worker simply handles the 25 ms interval at which the sequencer should update the event queue to enqueue the next series of events for playback. As this occurs outside of the main execution thread, this timer is less prone to "wander" under heavy load or on garbage collection.
The other Worker was invoked when the ScriptProcessorNode fires its onaudioprocess-callback. It receives an Array containing the event queue, and synthesizes its output into a Float32Array which is returned to the main application to be written into the AudioProcessingEvents outputBuffer.
The ScriptProcessorNode worker is now fully replaced with an AudioWorkletNode providing true threading.
A nice feature currently available only to Chrome users is the ability to connect MIDI peripherals to your computer and read / send MIDI messages from / to them from your web browser.
As the feature is still experimental and it is likely that the draft will change (not to forget whether it will be adopted by most browsers at all) libraries are scarce or outdated.
You can view more details about zMIDI here.
Languages and technologies used:
DSP / synthesis-related
Those who want to know what makes a Commodore 64 sound like a Commodore 64, you can easily view the contents of the worklet that renders the audio here. It's hardly a big secret, it's known as pulse width modulation!