Linguacorpus

Voice Activity Detection (VAD)

Short answer. VAD labels each frame of audio as speech or non-speech, usually every 10 to 30 ms. The simplest usable version combines two features: frame energy, which says something is there, and zero-crossing rate, which says whether it is tonal like voiced speech or broadband like noise. Energy alone fails on any loud non-speech sound. Add hysteresis — separate thresholds to enter and leave a speech run — or the output chatters on and off at every frame boundary.

How to do it

Framing is the first decision

VAD is a per-frame classifier, so frame length sets the resolution. A 25 ms window with a 10 ms hop is the common default: long enough to contain a couple of pitch periods of a male voice, short enough that a stop consonant is not smeared into its neighbours.

Frame the signal, then compute features per frame. Energy is the mean squared sample value in the frame; zero-crossing rate is the fraction of adjacent sample pairs that change sign.

Two features doing two different jobs

Energy answers "is there anything here". ZCR answers "what kind of thing is it". Voiced speech at 110 Hz has a zero-crossing rate near 0.014, because the waveform crosses zero about twice per pitch period. Broadband noise sits near 0.5, because it crosses zero on almost every sample.

This is why an energy-only VAD misfires on a chair scrape or a fan: those are loud, so the gate opens, and only the ZCR check rejects them. In the run below the noise burst is detected at -4.5 dB — louder than any of the speech — and is still rejected.

Hysteresis stops the output from chattering

With one threshold, a frame sitting at the boundary flickers. Speech has natural amplitude dips between syllables, so a single-threshold detector fragments one word into three.

Hysteresis fixes it with two thresholds: -35 dB to open a speech run, -45 dB to close it. Once open, a run stays open through quiet stretches, and the state only changes when the signal moves decisively. It is a five-line change that removes most spurious boundaries.

What VAD decides for everything downstream

Speech time as a fraction of total audio is a data specification, not a statistic. If you buy 100 hours and 40% of it is silence, you bought 60 hours of speech. VAD is how that gets measured, which is why a delivery should report speech ratio and not only total duration.

The same number drives ASR cost, diarization accuracy, and SNR measurement. Get the threshold wrong and every one of those inherits the error.

Code

A two-feature VAD with hysteresis on synthesized audio containing three speech regions and one loud noise burst.

import numpy as np

SR = 16000
rng = np.random.default_rng(7)
FLOOR = 0.0005


def speech_like(dur, seed):
    """Syllable-modulated voiced source: what the energy gate should keep."""
    r = np.random.default_rng(seed)
    t = np.arange(int(dur * SR)) / SR
    f0 = 110 + 40 * np.sin(2 * np.pi * 1.7 * t)
    x = np.sin(2 * np.pi * f0 * t) + 0.5 * np.sin(2 * np.pi * 3 * f0 * t)
    x *= 0.5 + 0.5 * np.sin(2 * np.pi * 4.0 * t)          # ~4 syllables per second
    return 0.3 * x + 0.02 * r.standard_normal(len(t))


def noise_burst(dur):
    """Broadband noise at a speech-like level: a fan, a chair scrape."""
    return 0.15 * np.random.default_rng(9).standard_normal(int(dur * SR))


def silence(dur):
    return FLOOR * rng.standard_normal(int(dur * SR))


audio = np.concatenate([speech_like(0.6, 1), silence(0.5), speech_like(1.2, 2),
                        silence(0.5), noise_burst(0.4), silence(0.5), speech_like(0.4, 3)])
win, hop = int(0.025 * SR), int(0.01 * SR)
frames = np.lib.stride_tricks.sliding_window_view(audio, win)[::hop]
db = 10 * np.log10((frames ** 2).mean(axis=1) + 1e-12)
db -= db.max()
zcr = (np.diff(np.sign(frames), axis=1) != 0).mean(axis=1)

# hysteresis: -35 dB opens a speech run, -45 dB closes it
state, speaking = np.zeros(len(db), dtype=bool), False
for k in range(len(db)):
    speaking = db[k] > (-35 if not speaking else -45)
    state[k] = speaking

edges = np.flatnonzero(np.diff(np.r_[0, state.view(np.int8), 0]) != 0)
print(f"{'run':>4}{'start':>8}{'end':>8}{'dur':>7}{'level dB':>10}{'ZCR':>8}  verdict")
speech_time = 0.0
for k in range(0, len(edges), 2):
    a, b = edges[k], edges[k + 1]
    voiced = zcr[a:b].mean() < 0.25
    speech_time += (b - a) * hop / SR if voiced else 0.0
    print(f"{k // 2:>4}{a * hop / SR:>8.2f}{b * hop / SR:>8.2f}{(b - a) * hop / SR:>6.2f}s"
          f"{db[a:b].mean():>10.1f}{zcr[a:b].mean():>8.3f}  "
          f"{'speech' if voiced else 'noise-like, rejected'}")

print(f"\nfile {len(audio) / SR:.2f}s   energy gate opened {state.sum() * hop / SR:.2f}s   "
      f"after the ZCR test {speech_time:.2f}s")
  • numpy only, and every sample is synthesized, so the script has no external inputs.
  • Four runs are detected. The three speech runs have ZCR 0.09 to 0.13 and are kept; the noise burst has ZCR 0.504 and is rejected despite being the loudest run at -4.5 dB.
  • The energy gate opens on 2.64 s of the 4.10 s file. After the ZCR check, 2.22 s of speech remains — the difference is the burst the energy gate alone would have kept.

Where this goes wrong

Using one threshold

A single-threshold VAD cuts words at every low-amplitude syllable. On a ten-second utterance you can get fifteen speech regions where there are three, and every downstream step that expects one region per utterance — normalization, alignment, duration filtering — inherits the fragmentation. Use hysteresis, with the close threshold 8 to 12 dB below the open threshold.

Assuming the silence between words is silent

In far-field and mobile recordings the pauses contain room tone, traffic and other talkers. A VAD tuned on studio audio marks all of it as speech, inflating both your speech ratio and your SNR estimate. Tune the thresholds on the noisiest condition in the dataset, not the cleanest.

Reporting speech ratio without saying how it was measured

Speech ratio depends on the frame length, the hop, the thresholds, and whether leading and trailing silence are counted. Two teams will report different ratios for the same file. Publish the settings with the number.

Treating VAD output as ground truth for annotation

If you pre-segment audio with a VAD and hand the segments to annotators, VAD errors become annotation errors and are invisible afterwards. A missed speech region is not a region someone forgot to annotate, it is a region nobody was ever asked about. Send a sample of VAD output for human review before it becomes the segmentation of record.

When to buy the data instead of building the pipeline

Build it — a usable VAD is under fifty lines and you should tune the thresholds on your own audio anyway. Buy when the segmentation has to be right rather than roughly right: word-level boundaries for a training corpus, or speech regions in a noisy far-field condition where a threshold rule cannot separate a distant talker from a nearby fan. That is boundary annotation, and it is priced by the hour like any other annotation. Ask for the speech ratio of any delivery you receive.

How buying training data works →

More technical reference

All technical reference →

Need data for Voice Activity Detection (VAD)?

Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline — not a range. If we cannot source it well, we will tell you that instead.

  • Pilot batch before the full run, so problems surface early.
  • Consent documentation delivered with the data.
  • No medical or clinical data. No recorded telephone calls.

We reply within two business days. Your details are used only to answer this request. See our privacy policy.

Contact

Talk to a human

Send a specification and we will come back with a real number and timeline.

Submit a sourcing request

Or email hello@linguacorpus.com