WebSID

Commodore 64 synthesizer inside the browser

What is it ?

WebSID is an in-browser synthesizer that approximates (purists might argue it's a poor "emulation" of) the sound of the Commodore 64, the popular home computer of the 1980's whose SID chip (either MOS 6581 or later MOS 8580 chips) defined much of what we now know as the "characteristic sound of retro videogames".

The chip had its fair share of limitations : you could only play three channels simultaneously, which led to the introduction of playing arpeggios (the fast, rippling sequence of notes) instead of multi-timbral chords, just so the remaining two channels would remain free to synthesize other sounds (e.g. to keep a bass-line and drum beat going).

You can jump straight to WebSID and start playing chiptune right now!

 

How did it come about ?

WebSID was initially created as a Chrome experiment to test-drive the possibilities of synthesizing audio inside a web browser in real-time using the experimental Web Audio API.

After the initial release gained popularity (after both Google as well as the audio community embraced the application), WebSID has been updated regularly to meet feature requests and introduce new functionality. WebSID currently features the following:

  1. Record, playback, save and share songs with friends
  2. Fully responsive, works on iOS and Android tablets and phones
  3. Connect MIDI keyboards to play WebSID using a keyboard (Chrome only)
  4. Works offline and is available for free as a Chrome app

So Web Audio... how does that work out ?

First up, Web Audio has nothing to do with the "HTML5" <audio>-tag. Web Audio provides an API to generate and manipulate audio in real time, all within the web browser. The API provides a plethora of modules out of the box, such as filters, oscillators, wave tables and even convolution reverb.

Though the API's modules are accessible in JavaScript, they actually run outside of the VM on the native layer of the CPU, providing high performing results. You can elegantly chain these "nodes" together to quickly sketch an aural idea.

While it is recommended to create your custom audio processors by combining these supplied modules, the ScriptProcessorNode is likely to be of most interest to those who wish to synthesize and generate their audio in real time using their own custom synthesis routines.

So how does this work ?

onAudioProcess

Audio is processed one buffer at a time in a ring buffer queue (consisting of two buffers, where one is currently playing back its contents, while the other is queued for playback, a familiar concept in digital signal processing). Each time a new buffer presents itself for enqueue-ing, the onaudioprocess-callback is fired on the ScriptProcessorNode. At this point the buffer is available in JavaScript (as part of the callbacks associated AudioProcessingEvent) and you can use JavaScript to fill its outputBuffer-property with audio.

Let me use this space to state that JavaScript (currently) feels utterly useless at calculating complex algorithms in real time like those used in signal processing and synthesis. The premise of running a script inside a VM, "aided" by garbage collection in a single thread makes this task very hard indeed (although it will be interesting to see what Audiotool can come up with next!).

Not all is bleak, as JavaScript also provides an elegant base for creating applications. So with paying a little attention, a lot of mileage can be gained just by knowing how to deal with the challenges, which brings us to:

WebSID "under the hood"

Sequencing and timing: the audioContext has the most stable and precise clock available to JavaScript, using time (in seconds) as a floating point value it is possible to enqueue events at a specific start time (relative to when the audioContext was created), enabling us to enqueue and play back events in a musical composition with the highest precision.

When playing one of WebSIDs keyboards, the notes are generated as a value object (VO) representing properties like frequency (the pitch of the note in Hz) and phase (used to generate the waveform). This object is added to an "event queue" which basically holds all the notes that should be synthesized into the enqueued output buffer.

When playing back a pre-recorded composition using the sequencer, the notes are enqueued by using temporary OscillatorNodes. These start their oscillation (silently!) at the currentTime of the audioContext and stop at the time the note should start playing, firing an ended-Event. Upon receiving the ended-callback we instantly create a VO for the note and add it to the event queue to be synthesized instantly. Basically, the OscillatorNodes are used solely as a highly accurate clock.

Sequencing the sequencer

At a regular interval, the sequencer checks which events (inside the currently playing/enqueued measure) should start their playback for the next timeframe. E.g.: every 25 milliseconds the sequencer checks the events that should start playing within now and the next 100 milliseconds and enqueues them using the "Oscillator clock"-trick mentioned above. Note the lookahead is larger than the interval to overcome notes dropping out due to drifting interval callbacks.

Optimization techniques

The garbage collector is your enemy. Seriously. Without using workers, this will eventually stall timers, queues, etc. even on fast machines. Even worse, OscillatorNodes that are created and aren't strongly referenced (thus become eligible for garbage collection as the creating functions body exits), will never fire their attached callback handlers after they are cleaned up. Which is ironic, given how most JavaScript memory leaks occur due to dangling listeners!

Apart from keeping strong references, Object pooling is highly beneficial as it overcomes unnecessary allocation and de-allocation of Objects, especially convenient when you need a lot of "temporary" Objects.

Web Workers

JavaScript is single threaded and as such the rendering of the UI might suffer during heavy calculations, as well as the timer accuracy of setInterval/setTimeout. To overcome this, WebSID uses two Workers that operate outside of the main execution thread.

Web Workers provide a means to run scripts in background threads, separated from the main thread that is responsible for rendering the user interface. The Workers operate in a different scope than the main thread with no access to the document, but for our purposes they are very useful (and without having to worry about mutual exclusion!).

One Worker simply handles the 25 ms interval at which the sequencer should update the event queue to enqueue the next series of events for playback. As this occurs outside of the main execution thread, this timer is less prone to "wander" under heavy load or on garbage collection.

The other Worker is invoked when the ScriptProcessorNode fires its onaudioprocess-callback. It receives an Array containing the event queue, and synthesizes its output into a Float32Array which is returned to the main application to be written into the AudioProcessingEvents outputBuffer.

WebMIDI

 

A nice feature currently available only to Chrome users is the ability to connect MIDI peripherals to your computer and read / send MIDI messages from / to them from your web browser.

As the feature is still experimental and it is likely that the draft will change (not to forget whether it will be adopted by most browsers at all) libraries are scarce or outdated.

As such I wrote a small adapter interface called zMIDI to transfer messages from connected musical hardware and make them available through JavaScript.

You can view more details about zMIDI here.

Third party components

Note that not everything in WebSID was built from scratch, so credit where credit is due: the following (excellent) libraries were integrated within WebSID:

PureMVC framework providing a solid publish-subscribe pattern.

Greensock Animation Platform used to add eye candy to the user interface.

Languages and technologies used:

JavaScript, Web Audio API, Web MIDI, Web Workers, HTML, CSS, PHP, MySQL.

DSP / synthesis-related

Those who want to know what makes a Commodore 64 sound like a Commodore 64, you can easily view the contents of the worker that renders the audio here. It's hardly a big secret, it's known as pulse width modulation!