Wake word data: what keyword spotting needs that ASR data does not

A detector, not a transcriber

A keyword spotting model does one thing: decide, frame by frame, whether a specific phrase was spoken. It runs continuously on a device with a small compute budget, and its output is a decision and a timestamp, not a transcript. The consequence for data follows immediately: a transcript is not the label. What the model needs is where the phrase starts and ends in the audio, and a large body of audio in which the phrase never appears.

The metric is different as well. Keyword spotting is measured in false accepts per hour of non-trigger audio at a fixed false-reject rate, not in word error rate. That number depends almost entirely on the negative audio behind it, which is why the negative set is not padding around the dataset. It is half the product.

Positives: many speakers, few repeats

The phrase itself is cheap to record, so the tempting construction is thousands of repetitions from a handful of voices. That produces a model which recognizes those voices, and it fails for the new user who is the entire reason the product exists. A few hundred speakers with a handful of repeats each generalizes; ten speakers with a thousand repeats does not.

Repeats should vary the things that vary in use, not the things that are convenient to vary in a studio. Distance from the microphone, speaking rate, loudness, and the position of the phrase inside a longer sentence all belong in the positive set, because in real use the trigger is rarely the first thing said and is often followed immediately by the command.

• The phrase in isolation, at a normal and at a fast rate.
• The phrase inside carrier sentences, at the start, the middle and the end.
• Soft speech, and speech from a distance where the level drops.
• The phrase from speakers of several accents, weighted toward the markets the product ships to.
• Child voices, if children will use the product. They are a known weak point for wake words.

Negatives are where the hours go

Negative audio is everything the model must learn to ignore, and it comes in three kinds. Confusables are near-misses: words that share phones with the trigger, and ordinary sentences that contain a similar-sounding sequence. General speech is conversation, podcasts and television. Non-speech is music, room tone, appliances and the other voice assistants in the room.

The confusable set should be mined rather than guessed. Run the current model over the negatives, collect the clips it fires on falsely, and add them back. A round of that shrinks the false-accept rate faster than adding generic speech does, because it teaches the exact boundary the model is failing at.

One rule applies to all three kinds: negatives must be recorded with the same devices, rooms and microphones as the positives. If the positives are clean studio audio and the negatives are noisy field recordings, the model learns the channel instead of the phrase, and it will fire on anything clean.

Write the deployment condition into the spec

The specification should describe the target device and the room, not only the language. A phone held to the ear, earbuds with a beamformer, and a speaker across a room produce different signals for the same phrase, and a model trained on one condition underperforms in another.

• Devices and microphone types, with the same set used for positives and negatives.
• Distance bands, and the signal-to-noise ratios that come with them.
• A padding convention: how much silence sits before and after each clip, applied consistently.
• The boundary convention for the phrase, marked at the word level or to the phone.
• The sample rate, which should match the front end the model will run behind.

Test what you bought before you trust it

A wake word evaluation has one shape: fix the false-reject rate, then measure false accepts per hour on held-out negative audio. Report the operating point every time, because a false-accept rate without one is not comparable to anything. The negative set has to contain the categories that actually break models: music with vocals, television dialogue, and clips where a similar phrase is spoken by a different voice.

Two leakage checks are worth running on delivery. No speaker should appear in both the positive and the negative set, because a model can learn a voice rather than a phrase. And the test negatives should come from the deployment condition rather than from the clean studio used for the positives. A wake word that scores well in the lab and fires at the television was tested on the wrong audio.