基于词典的汉藏句子对齐研究与实现

双语语料库加工的关键技术之一是对齐,构建句子级别的对齐语料是构建语料库最基本的任务。该文参考其他语言句子对齐的成熟的方法,针对藏文语言的特殊性,提出基于词典的汉藏句子对齐。整理了对齐所用双语词典,并对其词语覆盖率进行了评价。在汉藏句子对齐过程中发现汉语与藏文的分词粒度不同的问题,采用在藏汉词典中进一步查词并在汉语句子中比对的方法,使正确句对的得分增加,从而提高对齐正确率。采用该方法准确率为 81.11%。

Research and realization of Dictionary-Based Chinese-Tibetan Sentence Alignment

To construct the bilingual parallel corpus, the alignment at sentence level is a basic task. Considering the specific characteristic of Tibetan language, this paper proposes a dictionary-based Chinese-Tibetan sentence automatic alignment method. It establishes a bilingual dictionary for alignment, and evaluates its word coverage. To address the issueof different granularity between Chinese word segmentation and Tibetan word segmentation, this paper chooseto further look up the remaining big Tibetan word in Tibetan-Chinese dictionary and then match it in the original Chinese sentence, which increases the precision . Experiments show an average precision of 81.11% for this approach.
Key wordsChinese-Tibetan sentence alignment; dictionary; word segmentation granularity; parallel corpus; Tibetan information processing