Spectral Noise Reduction

A gate can silence the gaps between your words, but it can't touch the fan humming underneath them. Spectral noise reduction can: it slices your signal into hundreds of frequency bands, learns how much of each band is just the room, and subtracts the room — while you're still talking. Here's how a forty-five-year-old idea became the quiet workhorse of every voice chain.

Subtracting the room, one frequency at a time

The trick that makes this possible is a change of viewpoint. In the time domain, a voice and a fan are hopelessly intertwined — one waveform, sum of both. Transform a short slice of audio into the frequency domain, though, and they separate: the fan lives as a steady ridge of energy across specific frequencies, while speech is a fast-moving pattern that occupies different frequencies from one moment to the next. If you know how tall the fan's ridge is at every frequency — the noise floor — you can subtract that much energy from every slice, then transform back to a waveform. Speech loses almost nothing, because at any instant most of its energy towers far above the floor. The fan, which never rises above its own floor, simply vanishes.

That's the whole idea. Everything else in this article is about the two hard parts: knowing the floor without being told, and subtracting without leaving scars.

A short history: from Dolby to Boll

The analog ancestor of the idea is Dolby A (1966), Ray Dolby's professional tape noise-reduction system. It couldn't analyze frequencies individually, but it did the next best thing: it split the signal into four bands and treated each band's noise separately — the first mainstream acknowledgment that noise is a per-frequency problem, not a volume problem. Dolby SR (1986) pushed the same multiband philosophy much further, and between them they kept analog tape viable for decades.

True per-frequency subtraction had to wait for digital signal processing. The founding paper is Steven Boll's 1979 spectral subtraction: estimate the noise spectrum during pauses, subtract it from the short-time spectrum of the noisy speech, and resynthesize. It worked, and it also exposed the method's signature artifact — isolated tones flickering at random frequencies, which listeners nicknamed musical noise. Through the 1980s the estimation got smarter: Ephraim and Malah's MMSE estimators (1984) replaced blunt subtraction with a statistically optimal per-frequency gain, and their decision-directed smoothing dramatically tamed musical noise. The last big piece arrived with Rainer Martin's minimum statistics work in the 1990s (refined in 2001): a way to track the noise floor continuously, by observing that even during speech, each frequency band's power keeps dipping down to the noise floor whenever speech momentarily leaves that band. Track the minima and you know the floor — no pause detection, no "please be quiet for calibration" step.

Why it exists

Level-based tools can't solve this problem, even in principle. A gate or expander makes one decision for the whole signal: pass or attenuate. While you're speaking, it must pass — and the noise rides through with you. Broadband noise also can't be equalized away: a fan isn't at a frequency, it's smeared across many, most of them shared with your voice. Cutting those frequencies cuts you too.

Spectral noise reduction exists because it makes the decision per frequency. At 150 Hz, right now, is this mostly voice or mostly fan? At 3 kHz? Hundreds of independent decisions per frame mean the processor can remove the hum from underneath a vowel while leaving the vowel intact — something no single-band processor can do.

How it works: the STFT and the noise floor

Practical systems are built on the short-time Fourier transform (STFT). The signal is chopped into overlapping windows of roughly 10–50 ms, each window is tapered (to avoid edge artifacts) and transformed into a spectrum, processed, transformed back, and the overlapping pieces are added together. With the right window shape and overlap, an unprocessed signal comes out bit-identical — the framework itself is transparent, so any change you hear is the processing, not the plumbing.

Inside each frame, the algorithm needs the noise floor estimate. Minimum statistics supplies it: for each frequency bin, track the minimum power seen over the last second or so. Speech is spiky and intermittent in every band, so those minima trace out the steady floor beneath it. The estimate updates continuously, which means the system adapts when the noise changes — a heater switching on is learned within the tracking window and then subtracted like everything else.

Musical noise, and the fundamental limit

Subtract naively and you create the artifact Boll discovered. The noise floor estimate is an average, but actual noise fluctuates around its average — so after subtraction, random bins in random frames poke above zero and survive as brief, pure tones. The result sounds like faint birdsong or water trickling behind the voice: birdies. Three countermeasures, used together, keep them inaudible. Over-subtraction removes somewhat more than the estimated floor, so fluctuations fall below zero instead of above it. A spectral floor then clamps every bin to a small minimum rather than true zero, so nothing flickers against dead silence. And temporal smoothing of each bin's gain stops any bin from changing fast enough to chirp.

The honest caveat: all of this rests on one assumption — that the noise is statistically steady. Fans, hums, hiss, air conditioning: excellent. A dog barking, a door slamming, someone else talking: invisible to a minimum-statistics tracker, because they don't hold still long enough to be learned. Transient noise needs different tools — click suppression for impulses, AI noise removal for everything that merely sounds like not-you.

In DeskBroadcast

DeskBroadcast's noise reducer is a direct descendant of this lineage. It runs a 1024-point STFT with 50% overlap and √Hann windows — the standard transparent-reconstruction configuration — and learns the per-frequency noise floor by minimum statistics over two sub-windows of roughly 0.75 s each, so if your noise changes (the heat pump kicks in), the floor is re-learned within about 1.5 s. Subtraction uses the full anti-birdie toolkit: over-subtraction scaled by the strength slider, a spectral floor, and 50% temporal smoothing on the per-bin gains. The whole stage adds about 43 ms of latency. In the app it's described the way it behaves: it learns the room during quiet moments — fans, hums and washing machines fade after a few seconds — and you can watch it happen live on the spectrum analyzer as the floor drops out from under your voice. Steady tonal offenders like mains hum are exactly the kind of statistically stationary noise this stage eats for breakfast.