WORD ERROR RATE, CHARACTER ERROR RATE, AND PHONEME ERROR RATE IN ASR: WHAT EACH METRIC MEASURES AND WHERE THEY DIVERGE
Keywords:
Keywords: automatic speech recognition; word error rate; character error rate; phoneme error rate; grapheme-to-phoneme; Uzbek; agglutinative languagesAbstract
Word error rate (WER), character error rate (CER), and phoneme error rate (PER) measure different aspects of automatic speech recognition (ASR). Controlled Uzbek examples illustrate how orthographic variation can raise WER without changing PER, while loanwords and allophony can yield higher naive PER than CER. An Acceptance layer distinguishes permissible pronunciation variants from errors. These illustrative findings support complementary evaluation, with explicit documentation of G2P and acceptance criteria rather than population-level conclusions.
References
1.National Institute of Standards and Technology (NIST). (2022). OpenASR21 Challenge Results. NIST. https://www.nist.gov/itl/tted/mltg/openasr21-challenge-results
2.Bisani, M., & Ney, H. (2008). Joint-sequence models for grapheme-to-phoneme conversion. Speech Communication, 50(5), 434–451. https://doi.org/10.1016/j.specom.2008.01.002
3.Unicode Consortium. (2024). Unicode Common Locale Data Repository (CLDR), Version 46: Uzbek locale data and collation. https://www.unicode.org/cldr/charts/46/collation/uz.html
4.Levenshtein, V. I. (1966). Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics–Doklady, 10(8), 707–710. Translated from Doklady Akademii Nauk SSSR, 163(4), 845–848 (1965).