Building a grapheme-to-phoneme lexicon that holds up

A grapheme-to-phoneme lexicon is a file that says how each written word is pronounced: the word, then the sequence of sounds it is made of. It is the bridge between text and audio, and three kinds of work need one - hybrid speech recognition systems, forced alignment, and speech synthesis.

End-to-end models do not need a lexicon to train, which is why teams often discover the problem late. The moment the project needs phoneme-level output, pronunciation scoring, or an alignment, a lexicon is required, and by then the pipeline is already running without one.

Start from what exists, then define the phone set

For English, CMUdict has been available for decades and covers on the order of a hundred thousand words with stress marked. For most other languages the situation is worse: a small free dictionary, a list of uncertain quality derived from another project, or nothing at all. Whatever the starting point, the first artifact to create is the phone set - the closed list of symbols the lexicon is allowed to use.

• The symbol system: an ASCII set designed for English, an IPA set that is portable across languages, or an ASCII encoding of IPA. The choice matters less than making it once. IPA is the right answer if the project will ever cover more than one language.
• Stress. Whether it is marked, and how. Unmarked stress is acceptable for a recognizer and not acceptable for synthesis.
• Syllable boundaries. Include them only if a downstream component uses them. A boundary convention that is half-applied is worse than none.
• Silence and noise. Whether the lexicon contains entries for pauses, and whether those entries live in the same file or a separate one.

Generate the rest, and keep the source of every line

A dictionary covers the common words. The corpus contains names, products, and inflections it has never seen. Three generators are worth combining: a rule-based system, which is fast, explainable, and wrong in a patterned way; a neural model trained on the dictionary you already have, which handles unseen words better because it learns which letter contexts matter; and a second, independent system used only to cross-check the first.

The workflow that works: generate candidates, keep the entries where two systems agree without review, and send the disagreements to a person. For a few thousand words, checking the disagreements by hand is cheaper than debugging a lexicon later, because every wrong pronunciation becomes an alignment error or a mispronounced word downstream.

Mark the source of every entry - dictionary, rule, model, or human. Six months later, when a subset of the corpus is misbehaving, that field is what finds the few percent of entries that came from a generator rather than from a person.

Words with more than one pronunciation

The hard part of a lexicon is not the words with one pronunciation. It is the ones with several, and they fall into three groups that need different handling.

• Homographs whose pronunciation depends on meaning or part of speech. The classic English examples are the noun and verb forms of words like record, present, and close. The equivalent elsewhere is worse: characters with several readings in Chinese and Japanese, and words whose vowel changes with grammatical function in other languages. Context is required. The options are a part-of-speech tagger, a context-aware model, or a curated list of the few hundred most frequent cases. That curated list is a small share of the vocabulary and a large share of the errors, which is what makes it worth the effort.
• Regional and register variants: two accepted pronunciations of the same word. Keep one, or keep both with a marker, but never keep a variant that none of your speakers uses, because the aligner will choose whichever fits the audio and hand back a confidently wrong boundary.
• Connected-speech reductions: going to said as gonna, and the general reduction of unstressed vowels. These usually do not belong in the lexicon at all. Store the citation form and let the aligner absorb the difference. If both the citation form and the reduction are stored, mark which is which, or the aligner picks whichever fits and hides the fact that the audio and the text disagree.

Clean the input before it reaches the generator

A lexicon is a list of words, and a transcript is not a list of words. Numbers, dates, times, amounts, ranges, and symbols have to be expanded into words before lookup, which is the same normalization problem as the one on the recognition side, run in the opposite direction. Skipping it means every number is reported as unknown and the aligner fails on the sentences that contain one.

Acronyms need an explicit decision and there is no rule that gets it right. Some are read as words, some are spelled out letter by letter, and the same string can be both in different organizations, so a maintained list is the only reliable source. Proper nouns and product names are the long tail: they are the words the system will get wrong, they are the words the buyer cares about, and the only reliable source is asking the person who named the thing. Keep domain entries in a separate file so they can be re-applied after the general lexicon is regenerated.

Measure it on words that were not used to build it

Two numbers describe a lexicon: coverage, the share of the test vocabulary that has an entry, and accuracy. Check coverage first, because a missing entry is not a small error - it stops the pipeline.

For accuracy, phoneme error rate is the standard metric and it hides the case that matters most. A word whose phones are all wrong scores a full error on that word, which can still look small when averaged over a long sentence. Report word-level accuracy as well, split into words that were in the dictionary used to train the generator and words that were not. A lexicon that is nearly perfect on seen words and weak on unseen ones is a different product from one that is uniformly good, and the split number is what tells them apart.

Keep a held-out evaluation list of a few hundred words, including a set of homographs and a set of proper nouns, and re-run it every time the lexicon is regenerated. Add one structural check that catches the worst class of corruption: every symbol in every entry must exist in the phone set file, and no entry may contain a stray space or an empty field. Those two defects pass every visual inspection and break the aligner at first use.