Quality metrics for TTS training data: what to measure per file, per session, and per voice

TTS data is judged on the voice, not the words

A speech recognition corpus tolerates a lot. Noise, a second speaker, a stumbled word — the transcript can still be correct, and the model can still learn. A TTS corpus is different. The target is one clean, consistent voice across hours of material, and every defect the recording carries is a defect the model learns and reproduces. The quality question is not whether a clip is intelligible, but whether a listener would accept it as a sample of one professional speaker.

That shifts the metrics toward signal quality, consistency over time, and recording discipline. It also makes per-file gating practical: most TTS defects are detectable automatically, and the ones that are not are cheap to find in a listening pass.

Signal quality, measured rather than claimed

Each of these runs per file and yields a verdict, pass or fail, with a reason code attached. The thresholds are project decisions; the metrics are not.

• Noise level: measured on the quiet stretches of the file, not asserted from the studio description. Files below the project floor go to re-record rather than to cleanup.
• Reverberation: a proxy such as the ratio of early to late energy, or an estimated reverberation time. A dead room and a lively room produce voices that sound different, and mixing both inside one speaker teaches the model the room.
• Clipping and level: count samples at full scale, and check that gain is consistent across the corpus. A file that clips on plosives is not repaired by normalizing it afterwards.
• Hum and interference: a persistent tone at the mains frequency, hiss, a fan, a chair. Each has a cheap detection route — spectral peaks, a noise-floor check on silence — and each is easier to catch while the session is still running.
• Format discipline: one sample rate, one bit depth, one channel convention for the whole voice. Material upsampled to the target rate is not equivalent to material recorded at it.

Consistency across a voice

A voice recorded over weeks drifts. Pitch settles, pace changes, the speaker's health or the room changes. Measured per session and plotted, the drift is visible long before anyone hears it in a sentence.

• Pitch: median and range per session, from a standard pitch tracker. A session whose median sits far from the voice's own baseline will sound like a different take of the same person.
• Speaking rate: words per minute, across sessions and across the read material.
• Spectral character: the long-term average spectrum per session, which moves with microphone placement and room even when nothing else changed.
• Style mix: for expressive corpora, the share of each style within a session, so a "neutral" voice does not quietly turn out to be one fifth animated reads.

Pronunciation and text-audio correspondence

In read-speech TTS data the text is known in advance, which enables two checks a recognition corpus cannot run. Forced alignment scores the audio against the prompt word by word, and a poorly aligned word is a candidate mispronunciation, a skipped phrase, or a prompt error — before any human listens to it. A round-trip check transcribes the audio with a recognition model and compares the result against the prompt; the disagreements are the review queue, and each candidate gets a listener's decision rather than an automatic one.

The third check is prompt fidelity: whether the speaker actually read the sentence that was shown. Corpora assembled from sessions where the reader improvised, repeated a line, or read the wrong card carry text-audio mismatches that are far cheaper to catch at recording time than at training time. Whatever the acceptance bar for mispronunciations, the number to track is the rate per thousand words per batch, decided by a listener on a sample, so the bar is enforced by measurement instead of by impression.

Recording state: the defects that are not noise

The last class is behavioral. False starts, repeated lines, page turns, mouth clicks, audible breaths, a cough, a chair squeak mid-sentence. None of these fail a spectral check, and all of them are audible in synthesized output.

Handle them by labeling rather than deleting. Some of these events are wanted: breath and small pauses are part of what makes synthesized speech sound human, and a corpus with every breath surgically removed produces output that sounds mechanical. Tag each event type, let the training pipeline decide what to include, and keep the removal reversible.

Plan with the yield, not the raw hours

The number that turns all of the above into a plan is the clean-take yield: of the minutes recorded, what share passes every gate untouched, what share needs an edit, and what share is re-recorded. Track it by session and by reader. A falling yield for one reader is a fatigue or coaching signal; a falling yield across readers is usually a room or equipment problem.

Then put the yield into the arithmetic at the start. If the gates historically keep about two thirds of the raw material, the recording plan is sized on raw minutes, not delivered ones, and a schedule built on delivered hours will miss its date by the difference.