Linguacorpus

Character Error Rate (CER)

Short answer. CER is the same edit-distance calculation as WER with characters as the unit instead of words: CER = (S + D + I) / N, where N is the number of characters in the reference. It is the correct metric for languages written without spaces — Japanese, Thai, Chinese — where a word-level score has no boundaries to count. CER and WER are not convertible, because the tokenization that would convert them is the thing that was broken.

How to do it

The problem WER has with languages that do not space their words

Word error rate needs a word. Japanese and Thai do not put spaces between words, so a WER implementation has to guess where the boundaries are — and the simplest guess, treating the whole sentence as one token, makes every sentence either perfectly right or completely wrong. That collapses the metric to sentence accuracy while it is still reported as a word-level score.

The example below is one Thai sentence with a single character changed. WER reads 1.000, because the line is one token and it does not match. CER reads 0.100, which is the truth: one edit in ten characters. The second row adds three characters instead of changing one — WER still reads 1.000, while CER moves to 0.231. Two very different errors, one useless word-level score, two informative character-level ones.

What CER measures that WER cannot

CER runs on the character sequence directly, so it needs no tokenizer and no agreement about what a word is. That makes it stable across annotation conventions, which matters when two annotators disagree about whether a compound is one word or two — a disagreement that moves WER without a single acoustic error.

It also degrades gracefully. A hypothesis that is mostly right scores mostly right, instead of scoring 1.0 because the one token it had did not match.

The trap in Thai and other combining scripts

CER counts code points, not grapheme clusters. Thai vowels and tone marks are separate code points that render as part of the consonant they attach to. One visual character can therefore be two or three units in the denominator, and a system that gets the base consonant right but the tone mark wrong is charged one error out of three rather than one out of one.

This is not a reason to avoid CER. It is a reason to know that Thai CER is a slightly different measurement from English CER, and that a Thai CER of 0.05 is not the same achievement as an English CER of 0.05.

Pick one and stay with it

For space-delimited languages, report WER — it is what published baselines and your buyers use. For non-space-delimited languages, report CER and say so in the same sentence, because a reader who assumes WER will read your number as ten times better than it is.

  • Japanese, Chinese, Thai, Lao, Khmer, Burmese: CER.
  • Korean: either, because Korean uses spaces — but CER is still more stable across segmenters.
  • Mixed-script test sets: per-script numbers, never one pooled figure.

Code

Two Thai sentences — one with a single character changed, one with three characters added — scored both ways. WER returns the same 1.000 for both; CER separates them.

from jiwer import cer, wer

PAIRS = [
    ("one substitution", "อากาศดีมาก", "อากาศดีมาน"),
    ("three insertions", "วันนี้อากาศดี", "วันนี้อากาศดีมาก"),
]

for name, ref, hyp in PAIRS:
    print(f"{name}")
    print(f"  reference {len(ref)} code points, {len(ref.split())} whitespace token(s)")
    print(f"  WER {wer(ref, hyp):.3f}    CER {cer(ref, hyp):.3f}")
    print(f"  hypothesis {len(hyp)} code points\n")
  • pip install jiwer
  • Both references are a single whitespace token, so WER is 1.000 in both rows — it cannot tell a one-character error from a three-character one. CER reads 0.100 and 0.231, which is the difference the metric exists to show.
  • The Thai reference is 13 code points, not 12 visual characters, because the vowel and tone marks are separate code points. That is the combining-mark effect described above, visible in the denominator.

Where this goes wrong

Reading a CER as if it were a WER

A CER of 0.10 on Japanese is not comparable to a WER of 0.10 on English. A Japanese word averages two to three characters and an English word averages five, so the same CER implies a very different word-level quality. Name the metric in the same sentence as the number.

Scoring with a tokenizer you did not control

If your WER implementation segments Japanese with a dictionary-based tokenizer, the score depends on the tokenizer version. Two teams scoring the same hypothesis can disagree by several points. CER removes that variable, which is the main reason to prefer it here.

Forgetting that combining marks inflate the denominator

In Thai and Vietnamese a tone mark is its own code point. A system that drops tone marks can still post a low CER, because the marks are a small share of the total. Read the error breakdown, not just the ratio.

Comparing CER across scripts

CER on Chinese is not comparable to CER on Korean, because one unit carries a different amount of information in each. A Chinese CER of 0.05 and a Korean CER of 0.05 represent different amounts of error per unit of meaning, and pooling them into one figure makes that difference disappear. A mixed-script test set needs per-script numbers, not one pooled figure.

When to buy the data instead of building the pipeline

Build the scorer — it is the same fifteen lines as WER with a different unit. Buy the data when the language is the reason you cannot evaluate: a non-space-delimited language where you need human-verified reference transcripts before any CER number means anything, or a market where the same content has to exist in two scripts. Reference transcripts for these languages are a transcription product, and the useful version arrives with the segmentation convention documented, because otherwise you are measuring your annotators rather than your model.

How buying training data works →

More technical reference

All technical reference →

Need data for Character Error Rate (CER)?

Tell us the language, the hours, and what the data needs to look like. You will get a real number and a real timeline — not a range. If we cannot source it well, we will tell you that instead.

  • Pilot batch before the full run, so problems surface early.
  • Consent documentation delivered with the data.
  • No medical or clinical data. No recorded telephone calls.

We reply within two business days. Your details are used only to answer this request. See our privacy policy.

Contact

Talk to a human

Send a specification and we will come back with a real number and timeline.

Submit a sourcing request

Or email hello@linguacorpus.com