Linguacorpus

Noise-Robust Speech Recognition

Short answer. Noise-robust ASR is measured with a WER-versus-SNR curve, not a single WER number. Mix the same clean test set with the same noise at a ladder of signal-to-noise ratios — 20, 15, 10, 5, 0, -5 dB — and report the whole curve plus the noise type, because a system that wins at 10 dB in white noise can lose at 10 dB in babble. One SNR number is a point on a curve your deployment will not sit on, and the shape of the curve near your operating point is the only part that transfers.

How to do it

Define SNR on the active speech region, not the whole file

SNR is the ratio of speech power to noise power, and both halves need a definition. The mistake that matters is computing it over the whole file: if the noise recording contains a long lead-in of room tone and the speech recording starts with silence, mixing at "10 dB" by file-level RMS produces an effective SNR several dB away from 10 in the frames that carry speech, and the error direction depends on which file had more silence.

The fix is to compute the speech power over frames where speech is present — an energy threshold at 10% of the frame maximum is enough for the purpose — and the noise power over the noise file, then scale the noise so the ratio is exact. Print the achieved SNR next to the target for every mix. If they differ by more than 0.1 dB, your power estimate is wrong, and every downstream number inherits that error.

Build the ladder, and go below 0 dB

A curve needs at least four points, and the interesting ones are low. Above 20 dB most modern systems converge to within a point or two of their clean WER, so the top of the ladder is mostly a sanity check. From 10 dB down to 0 dB the systems separate, and below 0 dB the curve flattens toward the point where the model is guessing from context alone — a language model with no acoustic evidence still produces plausible text, which is why WER at -5 dB can be lower than you expect and why it is a poor place to compare models.

Include a clean or 40 dB reference point so the reader can see how much of the degradation is the model and how much is the noise.

Use at least three noise types, because they stress different things

Stationary noise (fan, HVAC, road hum) is broadband and constant, and it is the easiest case — a fixed spectral subtraction or a model trained with noise augmentation handles it well. Non-stationary noise (keyboard, dishes, a door) is bursty and violates the stationarity assumption that classical enhancement relies on. Competing speech (babble, a TV, another talker) is spectrally and temporally speech-like, so a model cannot separate it by spectral shape, and it is the hardest case for nearly every system.

Report the curve per noise type. Averaging three noise types into one curve hides the one that is failing, and the failing one is usually the deployment environment.

Hold out the noise recordings, not just the utterances

Augmentation with noise is standard, and the leak that follows is standard too. If the test noise files are the same files used for training augmentation, the model has effectively seen the test condition, and the curve comes out flatter than it deserves. Split by noise recording: the same physical fan, the same room, the same day should not appear on both sides. Report how many distinct noise recordings are in the test set, because a curve built on two noise files is a curve about those two files.

Summarize with the slope, not just the endpoints

The useful two-number summary is the clean WER and the degradation rate between 15 and 5 dB, in WER points per dB. Two systems with identical clean WER can differ by 3x in that slope, and the slope is what predicts behavior in a real room. A system that gains 8 WER points from 15 to 5 dB is more robust than one that gains 24, regardless of which is better in the quiet.

Code

Mix a synthetic speech signal with white and babble noise at six SNRs, verifying the achieved SNR, and print a WER-versus-SNR curve from a spectral-distortion proxy.

import numpy as np
import librosa

SR = 16000
rng = np.random.default_rng(0)

def speech_like(dur=4.0, seed=0):
    r = np.random.default_rng(seed)
    t = np.arange(int(SR * dur)) / SR
    f0 = 110 + 20 * np.sin(2 * np.pi * 0.7 * t)
    x = sum(np.sin(2 * np.pi * k * f0 * t) / k for k in range(1, 25))
    x = x * (0.5 + 0.5 * np.sin(2 * np.pi * 4.0 * t)) ** 2
    x = x * (np.sin(2 * np.pi * 0.6 * t) > -0.3)
    x = x + 0.002 * r.normal(size=t.size)
    return (x / np.max(np.abs(x))).astype(np.float32)

def mix_at_snr(x, n, snr_db, frame=400):
    """Scale the noise so that speech-active power / noise power = snr_db."""
    nf = len(x) // frame
    frames = x[:nf * frame].reshape(nf, frame)
    energy = (frames ** 2).mean(axis=1)
    active = energy > 0.1 * energy.max()
    speech_power = (frames[active] ** 2).mean()
    noise_power = (n ** 2).mean()
    g = np.sqrt(speech_power / (noise_power * 10 ** (snr_db / 10)))
    return x + g * n, 10 * np.log10(speech_power / (noise_power * g ** 2))

def proxy_wer(clean, noisy):
    """Spectral-distortion proxy, not a real WER. See the notes."""
    mc = librosa.feature.melspectrogram(y=clean, sr=SR, n_mels=40)
    mn = librosa.feature.melspectrogram(y=noisy, sr=SR, n_mels=40)
    active = mc.sum(axis=0) > 0.01 * mc.sum(axis=0).max()
    d = np.abs(np.log(mc[:, active] + 1e-8) - np.log(mn[:, active] + 1e-8)).mean()
    return 1.0 / (1.0 + np.exp(-(d - 0.55) * 9.0))

x = speech_like()
white = rng.normal(0, 1, x.size).astype(np.float32)
babble = (speech_like(seed=5) + 0.7 * speech_like(seed=6)).astype(np.float32)

for name, noise in (('white', white), ('babble', babble)):
    print('--- %s noise ---' % name)
    for snr in (20, 15, 10, 5, 0, -5):
        y, achieved = mix_at_snr(x, noise, snr)
        print('target %5.1f dB   achieved %5.1f dB   proxy WER %.3f'
              % (snr, achieved, proxy_wer(x, y)))
  • pip install numpy librosa soundfile. librosa is used for the mel spectrogram only.
  • Expected output: the achieved column should match the target to within 0.1 dB. If it does not, the speech-power estimate is picking up the pause frames and the whole curve is shifted.
  • The WER column is a proxy built from log-mel distortion, not a recognizer. It has the right shape — flat at the top, steep in the middle, flattening again at the bottom — and it is not a substitute for running a real model.
  • Note that babble does not come out systematically worse than white noise here. That is a limitation of the proxy, not a finding: babble hurts real recognizers because it is speech-like and confusable, which spectral distortion does not measure. Do not tune on a proxy.

Where this goes wrong

Mixing at file-level RMS

The noise file and the speech file rarely have the same proportion of silence, so scaling by whole-file RMS lands the effective SNR several dB off target, and in a direction that depends on which file had more quiet. Compute speech power over active frames only, and print the achieved SNR for every mix so the error cannot hide.

Reporting a single SNR point

"WER of 18% at 10 dB SNR" describes one condition on one noise file. Without the rest of the curve you cannot tell whether the system is on the steep part or the flat part, which is exactly what determines how it behaves when the real SNR drifts. The curve costs nothing extra once the mixing code exists.

Using white or synthetic noise as the only test condition

White noise is the easiest case for every method, including the cheap ones, so a system tuned on it looks better than it is and the tuning generalizes poorly. Real environments are dominated by low-frequency machinery, which is spectrally colored, and by other talkers. A test set with only white noise has measured the wrong thing, and the ranking it produces does not survive contact with a real room.

Reusing training noise files in the test set

Noise augmentation is the standard robustness technique, and it makes the test set a training set if the same recordings appear on both sides. The model recognizes the specific fan, not the class of fan. Split by physical recording, and state how many distinct noise recordings the test set contains — a curve over two files is a statement about two files.

Reporting WER at negative SNR as evidence of robustness

Below about -5 dB there is little acoustic information left, and a strong language model produces fluent, plausible text that happens to be wrong. WER stops tracking acoustic robustness and starts tracking the language model, which makes the numbers both surprisingly low and uninformative. Cut the curve where the model is still listening.

When to buy the data instead of building the pipeline

Buy when your deployment environment is one you cannot record in — a call center, a car cabin, a factory floor, a specific device — and you need matched noise at matched SNR rather than whatever you can find. Buy when you need babble specifically, since clean multi-talker recordings with speaker labels and transcripts are the expensive part and the part that decides whether the model survives the real case. Buy when the test set has to be held out properly, meaning distinct noise recordings and enough of them to make the curve meaningful. If you are doing early experimentation and just need to know whether your architecture is more robust than the baseline, generate the noise yourself and mix it; the ladder in this page is enough to answer that question and it costs nothing.

How buying training data works →

More technical reference

All technical reference →

Need data for Noise-Robust Speech Recognition?

Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline — not a range. If we cannot source it well, we will tell you that instead.

  • Pilot batch before the full run, so problems surface early.
  • Consent documentation delivered with the data.
  • No medical or clinical data. No recorded telephone calls.

We reply within two business days. Your details are used only to answer this request. See our privacy policy.

Contact

Talk to a human

Send a specification and we will come back with a real number and timeline.

Submit a sourcing request

Or email hello@linguacorpus.com