Statistical Romanization for Abugida Scripts:

Data and Experiment on Khmer and Burmese

Chenchen Ding1 Vichet Chea2 Win Pa Pa3 Masao Utiyama1 Eiichiro Sumita1

1ASTREC, NICT, Japan 2NIPTICT, Cambodia 3UCSY, Myanmar

1{chenchen.ding, mutiyama, eiichiro.sumita}@nict.go.jp2vichet.chea@niptict.edu.kh 3winpapa@ucsy.edu.mm

1 Introduction

Linguistically, romanization is the task of transform-ing a non-Latin writing system into Latin script. Itis a language-specific task in natural language pro-cessing (NLP). Despite standard romanization sys-tems, conventional transcription methods are ap-plied prevalently in many languages. The spellingsare thus inconsistent and complicated in some cases.Consequently, statistical approaches are required foran efficient solution on this task.

We focus on the name romanization for languagesusing abugida scripts. Different from an alpha-betic system, in an abugida system, consonant lettersstand for a syllable with an implicit inherent vowel,and diacritics modify consonant letters to change ordepress the inherent vowel. We designed annotationschemes to establish a precise, consistent, and mono-tonic correspondence between the two different writ-ing systems on grapheme level, through which vari-ous machine learning approaches are facilitated.

Specifically, we collect and manually align 7, 658and 2, 335 person name romanization instances forKhmer and Burmese (Myanmar), respectively. Bothlanguages are with limited resources and NLP stud-ies. Experimental results demonstrate that standardapproaches of conditional random fields (CRF) andsupport vector machine (SVM) supervised by themanually annotated data achieve high precision inromanization. The annotated data have been re-leased under a CC-BY-NC-SA license.1

2 Related Work

In engineering practice, the romanization task is astring-to-string transformation, which can be cast asa simplified translation task working on grapheme

1The Khmer data are available at http://niptict.

edu.kh/khmer-name-romanization-with-alignment-on-

grapheme-level/. The Burmese data are available at http:

//www.nlpresearch-ucsy.edu.mm/NLP_UCSY/name-db.html.Please contact the first author to obtain the data if theservers are unstable.

level rather than on word (or phrase) level with no(or few) reordering operations. Hence, general SMTtechniques can be facilitated once training data areprepared. The phrase-based SMT plays a role ofbaseline in recent workshops [1, 2], whereas neuralnetwork techniques provide further gains in perfor-mance [4, 5]. Although neural network-based, purestring-to-string approaches prove powerful on dif-ferent transliteration tasks, there is still room forimprovement, especially on tasks between differentwriting systems. For example, the Thai-to-Englishtask, which is similar to our task, has relatively poorperformance in NEWS 2015. The problem arounddiacritics in abugida is also stated in Kunchukuttanand Bhattacharyya [6]. General techniques may offeracceptable solutions overall, but specialized investi-gation and processing are required for further im-provement on tasks for specific languages. In the re-mainder of the paper, the Khmer and Burmese datawith manual annotation are described in Secs. 3 and4, respectively. Experiments are reported in Sec. 5,and Sec. 6 contains the conclusion and future work.

3 Khmer Data

We collect and annotate 7, 658 real Khmer name ro-manization instances. An example of a Khmer namealigned with its romanization is illustrated in Fig. 1.

A special feature of Khmer script is that there aretwo series of consonant letters, which have differentinherent vowels. Furthermore, diacritics representdifferent vowels when added to corresponding conso-nant series, which leads to a very complicated vowelsystem. Another feature of Khmer script is that thevertically stacked consonant letters are very com-mon. One reason is the abundant consonant clus-ters in phonology; another reason is the etymologi-cal spelling of Sanskrit (Pali) derived words. Thosestacked letters are not strictly based on the syllablestructure. Additionally (and more problematically),the virama, i.e., the diacritic used to suppress the in-herent vowel, is absent in Khmer script, which makesthe identification of onset and coda difficult.

言語処理学会 第23回年次大会 発表論文集 (2017年3月)

Copyright(C) 2017 The Association for Natural Language Processing. All Rights Reserved.　　　 　　 　　 　　　 　　　　　　　　　　― 234 ―

KOY CHANTRAកយ ចនទរា2

10

5

31 6

8

11

1

ក

K

2

O

4 5

ច

CH

6

ន

N

8

ទ

T

យ

Y

3

រ

R

10

A

11

.

7

.

9

.

A

.

.

Figure 1: A Khmer name composed of two words.The upper part is the raw string pair and the lowerpart is the grapheme-aligned pair. In the raw Khmerstring, the bold lines show the boundary of the un-breakable unit in writing and the thin lines distin-guish the components. The order of Khmer lettersis marked by numbers. 4 is the space; 7 and 9

are stacking operators. The “.” stands for insertedinherent vowels on the Khmer side and for a silentplaceholder on the Latin side.

As phonemic syllables and writing units do notmatch well in Khmer script. Consistent alignmentprinciples are thus difficult to establish if the sylla-bles or writing units are taken as atoms in processing.Furthermore, there are numerous types of syllablesand writing units, which are too complex for statis-tical model learning because of sparseness. Basedon these facts, we prefer a pure character-level align-ment, where standalone consonant letters, diacritics,and the invisible staking operator are separated andtreated equally as grapheme on the Khmer side.

A consequent problem is the inherent vowels, oncewe completely ignore the syllable structure. We can-not judge whether a standalone consonant standsfor only one consonant or contains a further inher-ent vowel, due to ambiguity in the writing system.We thus apply a scheme to insert a mark to repre-sent the inherent vowel for all the “bare” consonantletters (i.e., consonant letters without any diacrit-ics or stacking operators) to establish a consistentalignment. This insertion is thus decisive based onthe surface spelling, and the ambiguity on the pres-ence or absence of the inherent vowel is converted towhether the inserted mark is silent.

In the example of Fig. 1. the consonant lettersare 1 , 3 , 5 , 6 , 8 , and 10 , where 3 and 5

are bare consonant letters,2 after which the inher-ent vowel is inserted (noted by a dot here and in-dicated by a wedge without original index). Then

2 2 and 11 are diacritics for 1 and 10 , respectively,

and 6 and 8 are followed by staking operators. So thesefour consonant letters are not bare.

a character-by-character alignment can be estab-lished.3 According to our overall principles, Khmerconsonant letters are aligned to Latin consonantgraphemes; diacritics, including inserted inherentvowels, are aligned to Latin vowel graphemes; andany silent part is aligned to a placeholder.

4 Burmese Data

လင 1

23

4

5

61

2

4

5

6We collect and annotate 2, 335 realBurmese name romanization in-stances, including students and fac-ulty names in a university, publicfigure names and names from mi-nority nationalities in Myanmar.

Syllable are clear and integrated units in Burmesescript, which can be identified by rules, i.e., all dia-critics and consonant letters with a virama must beattached to the letter they modify [3]. As an exam-ple, a Burmese syllable composed of six charactersis illustrated in the beginning of the section, wherethe numbers indicate the order of the characters inthe composition. 1 and 4 are consonant letters,

and 2 , 3 , 5 , and 6 are diacritics. Specifically,

2 and 3 form consonant clusters with 1 , 5 is a

tone mark, and 6 is the virama to depress the in-

herent vowel of 4 to form the coda of the syllable.In order to provide an efficient interface for the

Romanization task, we step into the syllable struc-ture and further extract sub-syllabic segments, , i.e.,onset, rhyme, and tone, to achieve a precise and con-sistent correspondence between Burmese and Latinscripts. Fig. 2 is a diagram of the sub-syllabic seg-mentation scheme. It is relatively intuitive to dividea Burmese syllable into two parts, onset and rhyme,despite the difficulties introduced by medial conso-nants. The rhyme is not further dividable except into stripe tones annotated by explicit marks.

Specifically, four diacritics are used to representthe medial consonants, i.e., yapin, yayit, wahswe,and hahto to represent /-j-/, /-j-/,4/-w-/, and /-h-/,5

respectively. As shown in Fig. 2, yapin, yayit, andhahto are placed into the onset part, but wahsweinto the rhyme part. This scheme is adopted fortwo reasons. (1) The phonotactical constraints arestrict on yapin, yayit, and hahto, but loose on wah-swe. Hahto is only used on sonorants (nasals andapproximants) and yapin / yayit only on velars andbilabials.6 In contrast, wahswe can be freely com-bined with all consonant phonemes with the triv-

3As the task is to transform Khmer script to Latin script,the graphemes are not guaranteed to be single letters on the

romanization side, e.g., the 5 corresponds to CH here.4Yayit originally represents /-r-/ while in the modern stan-

dard Burmese the phoneme /r/ has been merged into /j/.5actually a voiceless sign, e.g., changing /n-/ to /n

˚-/.

6Yapin can also be combined with /l-/.

Copyright(C) 2017 The Association for Natural Language Processing. All Rights Reserved.　　　 　　 　　 　　　 　　　　　　　　　　― 235 ―

syllable

onset rhyme

toneinitial medial nucleus coda*

-j-, -r-*, -h-, -w-

Figure 2: Burmese sub-syllabic segmentation.

ial exception of /w-/.7 (2) Yapin, yayit, and hahtomay change the property of the initial consonantswhile wahswe may change the property of the nu-cleus vowels. Besides the voiceless marker of hahto,yapin / yayit palatalize velar consonants.8 Contrar-ily, wahswe may add a rounded feature to the fol-lowing nucleus vowels.9 The two issues affect con-ventional romanization spelling that multigraphs ofLatin letters are used to transcribe onset clusterswith yapin, yayit, and hahto, or wahswe-rhyme clus-ters. So, the scheme is the only feasible way to seg-ment Burmese onset and rhyme to achieve a mono-tonic and one-to-one and alignment with Latin scriptin conventional romanization.

Burmese has two codas, nasal and glottal, andthree tones, creaky, low, and high. The three tonescan be combined freely for open and nasal-ended syl-lables, but not for glottal ended ones. The glottalending thus may be regarded as a fourth tone in someanalysis. Further, the nasal ending can be regardedas a nasalization of nucleus vowels. Hence, the codais not a necessary component from an extreme view-point, which places the nasal ending into nuclei andthe glottal into tones. However, in the writing sys-tem, the nasal and glottal endings are representedby consonant letters with a virama. These “coda-letters” affect nucleus vowels, so that they are notcompletely detachable. As a result, only two tonemarks, i.e., the visarga (high) and aukmyit (creaky)are segmented in our scheme, as they are “pure” tonemarks,10 and any further segmentation would ruinthe integrated of the rhyme.

Specific transposition is required for the sub-syllabic segmentation. An examples is illustrated inFig. 3. For the onset-rhyme segmentation, an onsetpart with multiple components is coded in the orderof “C + yapin / yayit + wahswe + hahto”, where Cis the initial consonant.11 It is obvious that thesegmentation cannot be conducted under the cod-ing order once wahswe and hahto appear simultane-

7/Pw-/ may be argued in some references. The combinationappears marginally in borrowing words and interjections.

8e.g., /kj-/ is actually /c-/ or /tC-/9e.g., changing /-aP/ to /-UP/ and changing /-að/ to /-Uð/

10The visarga is usually not transcribed and aukmyit is in-consistently represented by a final t in Romanization.

11Multiple medial consonants for one initial consonant ispossible while yapin and yayit cannot appear simultaneously.

လ

la

လင

hlwint

= +

w

h

င

nga

t

+ + + +

လ

la

w

h

င

nga

t

လ

hl

င

win

t

wahswe-hahtoswapping

aukmyit-viramaswapping

+ +

rhyme toneonset

sub-syllabicsegmentation

Figure 3: Transposition before segmentation.

ously. Hence, a wahswe-hahto swapping is re-quired. For the rhyme-tone segmentation, one prob-lem is the order of the aukmyit and virama in nasal-ended creaky-toned syllables.12 As the standard or-der should be “aukmyit + virama” in coding,13 anaukmyit-virama swapping is required.

In addition to the transposition pre-processing, a“dummy rhyme” is inserted for all the “bare onsets”,identical to the insertion process applied on Khmer.

5 Experiment

We tested a state-of-the-art bi-directional LSTM-based RNN approach [8],14 as well as standardCRF15 and SVM16 approaches on our data.

The experiments were cross-validated. The RNNhandles the task in a sequence-to-sequence way oncharacter-level, without using any a priori knowl-edge, while CRF and SVM take advantage of thedesigned segmentation and manual alignment. Thefeatures for CRF and SVM were tokens up to tri-grams. The settings from the original paper wereused for the RNN.17 We evaluated the experimentalresults by two metrics: the accuracy of target tokenlabeling (TOK)18 and the accuracy of target strings(LAT) where BLEU [10] on Latin letters was used.

The experiments by LSTM-based RNN are con-ducted using an eight-fold cross-validation on ourdata without using manual alignment. The perfor-mance reaches .953 in terms of LAT on our Khmer

12As mentioned, glottal endings take no tones.13However, the swapped order may introduce no problem in

displaying, so both orders are used in daily typing.14https://github.com/lemaoliu/Agtarbidir15CRF++ [7, 11] at http://taku910.github.io/crfpp/16KyTea [9] at http://www.phontron.com/kytea/17Embedding size is 500, hidden unit dimension is 500, and

batch size is 4. AdaDelta is used for optimization with a decayrate ρ of 0.95 and an ε of 10−6.

18TOK cannot be applied to the RNN approach as the align-ment is not an explicit variable.

Copyright(C) 2017 The Association for Natural Language Processing. All Rights Reserved.　　　 　　 　　 　　　 　　　　　　　　　　― 236 ―

2-fold 4-fold 8-foldTOK .987 / .988 .988 / .989 .989 / .990LAT .974 / .977 .976 / .978 .977 / .979

Table 1: Results on Khmer data.

2-fold 4-fold 8-foldTOK .946 / .944 .948 / .947 .947 / .947LAT .912 / .907 .913 / .911 .913 / .910

Table 2: Results on Burmese data.

data, which is an acceptable results. While RNN ap-proach cannot performed well on our Burmese data,where LAT is no more than .718. We find the RNNactually generates “Burmese-styled” Latin transcrip-tions but inaccurate. We thus consider the perfor-mance to be reasonable and attribute the causes forthe result to (1) the data size, which is still insuffi-cient to support an RNN model (less than one thirdof the Khmer data), and (2) the mapping betweenBurmese and Latin scripts is complex, where many-to-many alignment between characters is commonand is difficult to model without any heuristics.

Similar to RNN experiments, CRF and SVM ex-periments are also cross-validated, where the eight-,four-, and two-fold results are tested for compari-son. The results by CRF and SVM on Khmer andBurmese data are listed in Tables 1 and 2, respec-tively. The performance of the two approaches aresimilar: TOK is around .99 and LAT is around .98 onKhmer data; TOK is around .95 and LAT is around.91 on Burmese data. The performances outperformthose of RNN on both data sets, respectively. Weconclude that the romanization task does not re-quire features in a long distance, which RNN canmodel well, but precise local alignment provides use-ful and efficient information contributing to the per-formance. Therefore, we consider that it is the highquality of our annotated data, rather than a sophis-ticated model, that contributes more to the task.

As to engineering issues in practice, it takes hoursto train an RNN model on several thousand instancesin eight-fold cross-validation, while to train the SVMmodel in KyTea only takes seconds. Therefore, weconsider RNN to be a superfluous approach for theKhmer romanization task.

6 Conclusion and Future Work

We focused on the Romanization tasks on Khmerand Burmese, from the data preparation to experi-ments and discussion of statistical approaches. Thedata prepared in this study have been released un-der a CC-BY-NC-SA license to promote the researchof low-resourced language processing.

Acknowledgements

We thank Dr. Lemao Liu for his help in the RNNexperiments. We thank Mr. Kaing Hour for his helpin checking the Khmer data.

References

[1] Rafael E. Banchs, Min Zhang, Xiangyu Duan,Haizhou Li, and A. Kumaran. Report of NEWS2015 machine transliteration shared task. InProc. of NEWS, pages 10–23, 2015.

[2] Marta R. Costa-jussa. Moses-based official base-line for NEWS 2016. In Proc. of NEWS, pages88–90, 2016.

[3] Chenchen Ding, Ye Kyaw Thu, Masao Utiyama,and Eiichiro Sumita. Word segmentation forBurmese (Myanmar). ACM Transactions onAsian and Low-Resource Language InformationProcessing, 15(4):22, 2016.

[4] Andrew Finch, Lemao Liu, Xiaolin Wang, andEiichiro Sumita. Neural network transductionmodels in transliteration generation. In Proc.of NEWS, pages 61–66, 2015.

[5] Andrew Finch, Lemao Liu, Xiaolin Wang, andEiichiro Sumita. Target-bidirectional neuralmodels for machine transliteration. In Proc. ofNEWS, pages 78–82, 2016.

[6] Anoop Kunchukuttan and Pushpak Bhat-tacharyya. Data representation methods anduse of mined corpora for Indian languagetransliteration. In Proc. of NEWS, pages 78–82, 2015.

[7] John Lafferty, Andrew McCallum, and Fer-nando Pereira. Conditional random fields:Probabilistic models for segmenting and label-ing sequence data. In Proc. of ICML, pages 282–289, 2001.

[8] Lemao Liu, Andrew Finch, Masao Utiyama,and Eiichiro Sumita. Agreement on target-bidirectional LSTMs for sequence-to-sequencelearning. In Proc. of AAAI, pages 2630–2637,2016.

[9] Graham Neubig, Yosuke Nakata, and ShinsukeMori. Pointwise prediction for robust, adapt-able Japanese morphological analysis. In Proc.of ACL-HLT, pages 529–533, 2011.

[10] Kishore Papineni, Salim Roukos, Todd Ward,and Wei-Jing Zhu. Bleu: A method for au-tomatic evaluation of machine translation. InProc. of ACL, pages 311–318, 2002.

[11] Fei Sha and Fernando Pereira. Shallow parsingwith conditional random fields. In Proc. of HLT-NAACT, pages 134–141, 2003.

Copyright(C) 2017 The Association for Natural Language Processing. All Rights Reserved.　　　 　　 　　 　　　 　　　　　　　　　　― 237 ―