Linguacorpus

Audio Augmentation

Short answer. The three augmentations that pay off for speech recognition are additive noise mixed at a target SNR, small time shifts, and speed perturbation. All three preserve the transcript, which is what makes them cheap: no re-annotation is needed. The catch is that speed perturbation changes the duration and time shifts move the start offset, so the manifest has to be rewritten even though the text does not change. Augment and update the manifest in the same pass, not as an afterthought.

How to do it

Why these three and not others

An augmentation is useful when it simulates a condition the model will meet and does not change what the words are. Additive noise simulates a different environment. Speed perturbation simulates a different speaking rate, which is one of the largest sources of variation between speakers. Time shifts are weak on their own but stop a model from using absolute position as a cue.

Augmentations that change the transcript — pitch shifting far enough to alter vowel identity, or clipping heavy enough to destroy a word — are not augmentations. They are label noise.

Mixing to a target SNR, not a target gain

Adding noise at a fixed amplitude produces a different effective SNR for every utterance, because utterances differ in level. A quiet recording gets buried and a loud one is barely touched.

The correct operation is to scale the noise so the speech-to-noise ratio hits a target: measure the mean power of both, compute the gain that gives the ratio you want, apply it. The result is a corpus where a "10 dB SNR" condition really is 10 dB across the set.

Speed perturbation is the one that breaks things

Resampling to change speed shifts every frequency by the same factor, which is what makes it sound like a faster speaker rather than a pitch-shifted one. It also scales the duration by the inverse of the rate: a 1.1x rate turns 1.50 s into 1.36 s.

That duration change has to be written back to the manifest. Every downstream tool that trusts the manifest — alignment, segmentation, SNR measurement, duration-based filtering — is wrong about that file until it is. This is the single most common bug in augmentation pipelines.

Verify that the augmentation preserved the content

The transcript is unchanged by construction, so checking it proves nothing. What you can check is that the audio still carries the same envelope: compute a coarse energy fingerprint, correlate it against the original, and read the number next to the operation that produced it.

A time shift legitimately lowers the correlation because it moves the signal, and speed perturbation changes the time axis. In the table below, the unshifted variants sit above 0.99, the shifted ones at 0.87 and 0.94, and both are correct behaviour rather than defects.

Code

Apply the three standard augmentations to one synthetic utterance and report duration, sample count and an energy fingerprint for each variant.

import numpy as np
from scipy.signal import resample_poly

SR = 16000
rng = np.random.default_rng(3)
t = np.arange(int(1.5 * SR)) / SR
f0 = 120 + 20 * np.sin(2 * np.pi * 2.5 * t)
speech = np.sin(2 * np.pi * np.cumsum(f0) / SR) * np.hanning(len(t)) ** 0.5
TRANSCRIPT = "the transcript does not change"


def add_noise(x, snr_db, rng):
    noise = rng.standard_normal(len(x))
    return x + noise * np.sqrt((x ** 2).mean() / (noise ** 2).mean()) * 10 ** (-snr_db / 20)


def time_shift(x, ms):
    k = int(ms / 1000 * SR)
    if k > 0:
        return np.concatenate([np.zeros(k), x])[:len(x)]
    return np.concatenate([x, np.zeros(-k)])[-len(x):]


def speed_perturb(x, rate):
    return resample_poly(x, 100, int(round(100 * rate)))


def fingerprint(x, bins=64):
    hop = len(x) // bins
    e = np.array([(x[i * hop:(i + 1) * hop] ** 2).mean() for i in range(bins)])
    e -= e.mean()
    return e / (np.linalg.norm(e) + 1e-12)


variants = [
    ("original", speech),
    ("add noise, 10 dB SNR", add_noise(speech, 10, rng)),
    ("shift +120 ms", time_shift(speech, 120)),
    ("shift -80 ms", time_shift(speech, -80)),
    ("speed 0.9x (slower)", speed_perturb(speech, 0.9)),
    ("speed 1.1x (faster)", speed_perturb(speech, 1.1)),
]

base = fingerprint(speech)
print(f"transcript carried through every variant: {TRANSCRIPT!r}\n")
print(f"{'variant':<24}{'duration':>11}{'samples':>10}{'envelope corr':>15}")
for name, x in variants:
    print(f"{name:<24}{len(x) / SR:>10.3f}s{len(x):>10}{float(base @ fingerprint(x)):>15.3f}")
  • pip install numpy scipy
  • The speed variants change both duration and sample count — 1.50 s becomes 1.667 s at 0.9x and 1.364 s at 1.1x. Those are the values that must be written back to the manifest.
  • The fingerprint correlation separates the operations: above 0.99 for noise and speed, 0.874 and 0.943 for the two shifts. Read it against the operation, not against one global threshold.

Where this goes wrong

Augmenting without updating the manifest

Speed perturbation changes duration and time shifts move the start offset. If the manifest still holds the original values, alignment, segmentation and duration filters all silently operate on wrong numbers. Write the new duration in the same pass that writes the new audio.

Adding noise at a fixed gain

A fixed gain gives a different SNR for every utterance, so your noisy condition becomes a range rather than a setting. Mix to a target SNR and verify it after mixing.

Shifting audio across the utterance boundary

A time shift that pushes samples past the end of the buffer must pad or wrap. Wrapping moves the start of the utterance to the end and produces audio that does not match the transcript at all. Pad with zeros.

Augmenting the test set

Augmentation is a training-time technique. Adding augmented copies to the evaluation set makes the score depend on the augmentation recipe and hides the condition mismatch you were trying to measure.

Applying every augmentation at maximum strength

Speed perturbation at 1.4x changes phoneme durations enough that a model trained mostly on it degrades on normal speech. A model fine-tuned on a corpus where every file is perturbed at the maximum rate can end up worse on normal speech than the model you started with. Keep the range narrow — 0.9 to 1.1 is standard — and mix augmented and unaugmented copies.

When to buy the data instead of building the pipeline

Build it — these are twenty-line functions and the SNR range and speed range should be tuned against your own evaluation set. Buy when the condition you need to simulate cannot be synthesized from your existing audio: real far-field recordings from a real room, genuine background babble in a specific language, or a channel that only exists on one telephone network. Simulated noise teaches a model to ignore stationary noise; real noise teaches it to ignore the noise your users actually have.

How buying training data works →

More technical reference

All technical reference →

Need data for Audio Augmentation?

Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline — not a range. If we cannot source it well, we will tell you that instead.

  • Pilot batch before the full run, so problems surface early.
  • Consent documentation delivered with the data.
  • No medical or clinical data. No recorded telephone calls.

We reply within two business days. Your details are used only to answer this request. See our privacy policy.

Contact

Talk to a human

Send a specification and we will come back with a real number and timeline.

Submit a sourcing request

Or email hello@linguacorpus.com