Audio Splitter ProHow AI music source separation works
Taking a finished song and pulling it back into separate instruments sounds almost impossible. After all, the sounds have been added together. Yet modern AI models do it surprisingly well. Here is the idea without the heavy maths.
The problem
A stereo song is a single stream of numbers: the air pressure for the left and right speakers, 44,100 times per second. Every instrument has been added into those numbers. There is no mathematical way to subtract one sound exactly, because we do not know what it looked like on its own.
The idea: learn what instruments look like
Researchers train a neural network on a large collection of songs for which the separate stems are known. The model is shown the mix and asked to predict the stems. When it is wrong, its internal settings are nudged slightly. After many repetitions it learns the typical patterns of a voice, a kick drum or a bass guitar, and how they overlap.
Two ways of looking at sound
Some models look at the waveform, the raw signal over time. Others look at the spectrogram, a picture of which frequencies are present at each moment. The model used here, a Hybrid Transformer Demucs, uses both views and a transformer to compare different parts of the song with each other. A drum pattern that repeats, for example, helps the model recognise the drums every time they appear.
Why the song is processed in pieces
The network works on a window of about eight seconds. A long song is cut into overlapping windows, each is separated, and the results are blended together with a gradual crossfade so you never hear a seam. More overlap means smoother joins and a slower job. This is what the quality setting controls.
Why it runs in your browser
The model is converted to a portable format called ONNX and executed by a runtime compiled to WebAssembly. That lets the browser do the heavy calculation on your own processor, so your music never has to leave your device. The first visit downloads the model once and keeps it in browser storage.
Limits you should know about
- The result is an estimate, not the original studio stems.
- Very similar sounds, like a lead vocal and a harmony singing the same notes, can be confused.
- Phones have less memory and speed than computers, so long songs take longer.
Where the technology is going
Separation keeps improving, with models that split more instruments and make fewer artefacts. Running such models locally, privately and for free is becoming normal, and this site is one small example.