Linguacorpus

Speaker Verification

Short answer. Speaker verification decides whether two recordings are the same person. The standard approach turns each recording into a fixed-length embedding and compares them by cosine similarity, accepting the pair when the score passes a threshold. The two error rates trade off: raising the threshold rejects more impostors and more genuine speakers. The threshold where the two rates are equal is the equal error rate, and it is the usual single-number summary.

How to do it

The embedding is the whole trick

A recording is thousands of samples and a comparison needs a vector of a few hundred numbers. The embedding compresses the recording into a representation that keeps speaker identity and discards everything else — the words, the channel, the background.

The classic weak version is the mean of the MFCC vectors over time. It works because the average spectral envelope reflects vocal tract shape, which differs between people. It is weak because it also reflects the microphone: two recordings of the same person on different devices can sit further apart than two different people on the same device.

Thresholds are a trade, not an answer

A verification system outputs a score. Turning that into a yes or no requires a threshold, and the threshold sets two error rates: the false reject rate, which is genuine speakers turned away, and the false accept rate, which is impostors let in.

The two move in opposite directions. In the run below, moving the threshold from 0.735 to 0.935 takes false accepts from 25.8% to 0% while false rejects climb from 0% to 36.7%. Neither end is right. The right threshold is set by the cost of each error in the application, and those costs are rarely equal.

EER, and what it is good for

The equal error rate is the point where the two curves cross, which here is 4.8% at a threshold of 0.835. It is a single number that summarizes the system without committing to an operating point, which makes it useful for comparing embeddings.

It is not a deployment setting. Nobody deploys at the EER threshold, because the two errors almost never cost the same. A bank authenticating a payment and a phone unlocking for its owner have opposite priorities.

Why this example is easier than reality

The demonstration uses synthetic voices and a mild channel tilt, and it separates well. Real speaker verification with mean-pooled MFCC embeddings and genuine channel mismatch lands far worse, typically in the 5 to 15% EER range, and that gap is why the field moved to learned embeddings.

Modern systems use x-vectors or ECAPA-style networks trained on thousands of speakers with explicit augmentation for channel and noise. The evaluation method does not change — cosine similarity, a threshold, FAR and FRR, EER — only the quality of the embedding does.

Code

Three synthetic speakers with distinct formants, MFCC-mean embeddings under a random channel tilt, and the full FAR/FRR tradeoff table plus the EER.

import numpy as np
import librosa
from scipy.signal import lfilter

SR = 16000
SPEAKERS = {                                    # pitch, then three formants (Hz, bandwidth)
    "spk_a": (120.0, [(500, 110), (1600, 160), (2600, 210)]),
    "spk_b": (150.0, [(560, 120), (1750, 170), (2700, 220)]),
    "spk_c": (135.0, [(470, 105), (1500, 150), (2500, 200)]),
}


def utterance(f0, formants, seed, dur):
    """Source-filter: a harmonic stack shaped by the formant resonances."""
    r = np.random.default_rng(seed)
    t = np.arange(int(dur * SR)) / SR
    phase = 2 * np.pi * np.cumsum(f0 * (1 + 0.02 * np.sin(2 * np.pi * 3.3 * t))) / SR
    x = np.zeros_like(t)
    for k in range(1, 41):
        if k * f0 > SR / 2:
            break
        x += (sum(np.exp(-((k * f0 - fc) ** 2) / (2 * (bw / 2.355) ** 2))
                  for fc, bw in formants) * np.sin(k * phase) / k)
    x *= 0.5 + 0.5 * np.sin(2 * np.pi * 3.0 * t)
    return x / (np.abs(x).max() + 1e-9) + 0.002 * r.standard_normal(len(t))


def embed(x, tilt=0.0):
    """A one-pole spectral tilt stands in for a different microphone, then MFCC 1..19."""
    a = float(np.clip(tilt, -0.95, 0.95))
    x = lfilter([1.0 - a], [1.0, -a], x)
    m = librosa.feature.mfcc(y=x.astype(np.float32), sr=SR, n_mfcc=20)[1:]
    return (m / (m.std(axis=1, keepdims=True) + 1e-9)).mean(axis=1)


enrol = {s: embed(utterance(f0, fo, 100 + i, 1.5))
         for i, (s, (f0, fo)) in enumerate(SPEAKERS.items())}
rng = np.random.default_rng(0)
genuine, impostor = [], []
for i, (s, (f0, fo)) in enumerate(SPEAKERS.items()):
    for trial in range(40):
        probe = embed(utterance(f0, fo, 1000 + i * 500 + trial, 0.3), rng.normal(0, 1.5))
        for target, vec in enrol.items():
            score = float(probe @ vec / (np.linalg.norm(probe) * np.linalg.norm(vec)))
            (genuine if target == s else impostor).append(score)

genuine, impostor = np.array(genuine), np.array(impostor)
print(f"genuine  n={len(genuine)}  mean {genuine.mean():+.3f}  lowest {genuine.min():+.3f}")
print(f"impostor n={len(impostor)}  mean {impostor.mean():+.3f}  highest {impostor.max():+.3f}")
print(f"\n{'threshold':>10}{'false reject':>14}{'false accept':>14}")
for t in np.linspace(0.735, 0.935, 7):
    print(f"{t:>10.3f}{(genuine < t).mean():>13.1%}{(impostor >= t).mean():>14.1%}")

rates = [((genuine < t).mean(), (impostor >= t).mean(), t) for t in np.linspace(-1, 1, 801)]
frr, far, thr = min(rates, key=lambda r: abs(r[0] - r[1]))
print(f"\nEER {(frr + far) / 2:.1%} at threshold {thr:.3f}")
  • pip install numpy librosa scipy
  • Genuine scores average +0.927 with a lowest value of +0.763. Impostor scores average +0.599 but reach +0.855 — the overlap between those two ranges is what forces a threshold trade rather than a clean cut.
  • The table spans false accept 25.8% down to 0.0% while false reject rises from 0.0% to 36.7%. The EER is 4.8% at threshold 0.835.

Where this goes wrong

Reporting EER as the operating point

EER is a summary for comparing systems, not a threshold to ship. The threshold you deploy is set by which error costs more, and it will not be the EER point.

Enrolling and testing on the same recording

If the enrollment sample and the test sample are the same audio, the score is trivially high and the evaluation measures nothing. They must be different recordings, ideally from different sessions and devices, and the trial list should say which is which.

Ignoring channel mismatch

An embedding evaluated on recordings from one microphone degrades sharply when the probe comes from another. A system measured only on same-channel trials can show an EER several times better than the same system measured across channels, and the same-channel figure is the one that gets quoted. Split trials into same-channel and cross-channel and report both, because cross-channel is the number that predicts deployment.

Using a mean-pooled MFCC embedding in production

It is the right thing to write for a demonstration and the wrong thing to ship. Its EER on real cross-channel data is several times worse than a trained embedding, and the difference is not tunable away.

Balancing the trial list by accident

The ratio of genuine to impostor trials changes the reported FAR and FRR at any threshold. Fix the ratio deliberately, state it, and keep it constant across the systems you compare.

When to buy the data instead of building the pipeline

Buy when you need speakers rather than hours: verification depends on how many distinct people are in the data and how many sessions each appears in, and a corpus of 500 speakers with three sessions each is a different asset from 50 speakers with thirty sessions. Buy when the channels have to vary, because cross-channel trials are the ones that predict deployment and staging them requires real devices. Build it yourself when you are prototyping the pipeline and can generate or reuse a small set of speakers.

How buying training data works →

More technical reference

All technical reference →

Need data for Speaker Verification?

Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline — not a range. If we cannot source it well, we will tell you that instead.

  • Pilot batch before the full run, so problems surface early.
  • Consent documentation delivered with the data.
  • No medical or clinical data. No recorded telephone calls.

We reply within two business days. Your details are used only to answer this request. See our privacy policy.

Contact

Talk to a human

Send a specification and we will come back with a real number and timeline.

Submit a sourcing request

Or email hello@linguacorpus.com