Language identification data: the segment length decides the job

Two jobs hide behind one name

Language identification answers a narrow question: which language is being spoken in this segment. The answer is used in two very different places. A streaming router decides within the first moments of audio, before a recognizer commits to a language model. A batch pass sorts an archive of unknown recordings, where a longer listen is available.

The same label set serves both, but the data does not. A model trained and tested on long segments looks strong and then fails as a router, because accuracy falls as the segment shortens, and the fall is steep for closely related languages. Report accuracy per duration bucket — one second, three seconds, ten seconds, thirty seconds — instead of one pooled number. The pooled figure is an average over a curve, and the router lives at the bad end of it.

Fix the label set before anything is collected

The first decision is whether the classes are languages or varieties. Is Egyptian Arabic a class of its own or a slice of Arabic? Is the Spanish of Latin America separate from the Spanish of Spain? Merging classes afterwards is a mapping over labels and costs almost nothing. Splitting them afterwards is impossible, because the audio was never labeled at that depth. So collect the finest distinction that might be needed and record the parent class beside it.

Two related decisions belong in the same document. Whether an utterance that mixes two languages carries both labels, and what share of a second language makes it mixed. And the fact that some pairs are not separable from speech at all: Hindi and Urdu differ in script and in formal vocabulary, but everyday speech is close enough that a model will confuse them, and no label scheme changes that.

The confusable pairs decide whether the model is useful

LID errors concentrate in a small set of pairs that share phonology, vocabulary, or both. Spanish and Portuguese. Indonesian and Malay. Czech and Slovak. Danish, Swedish and Norwegian. Russian and Ukrainian. Dutch and Afrikaans. Thai and Lao. Mandarin and Cantonese. Xhosa and Zulu. The Arabic dialects against Modern Standard Arabic.

Coverage for a pair is not more hours of each language in general. It is speakers who make the pair distinguishable: multiple speakers per class, from regions where the variety is typical, and no shared speakers across the two classes in the training pool. A bilingual speaker recorded under both labels makes both classes fuzzier, which is a real-world condition worth its own evaluation slice but a poor basis for training.

Short segments punish everything else that is wrong

At one to three seconds the model is working from phonotactics and a few prosodic cues, and conditions that are merely annoying for long segments become decisive. A telephone-band recording at eight kilohertz removes the high-frequency cues that separate some pairs. Music and singing carry no language at all and still get classified. Background speech from a television puts two languages in one clip, and the label has to say whose language it is.

Two spec details matter more at short durations than anywhere else. Whether the clip was trimmed of leading silence, because a second of silence inside a two-second clip is half the evidence gone. And the minimum clip length accepted into the set, because a duration distribution with a long tail of very short clips drags the measured accuracy down in a way that the histogram explains and a single number does not.

Spec fields and an evaluation that matches the router

The fields that make an LID delivery checkable: hours and distinct speakers per language, the duration histogram rather than a mean, the channel, and the policy for mixed utterances. Speakers per language is the field most often missing and the one that predicts generalization.

Evaluation should mirror the deployment. If the model will route, score it on short segments and report per-language recall together with the confusion matrix, not accuracy alone. Use a macro average so the small languages stay visible, and read the matrix at the confusable pairs first. A routing mistake is not a small error — a wrong language sends the audio to a recognizer that cannot read it, and the whole utterance is lost — so the pairs where the model hesitates are the number that matters.