|
|||||
|
|
| 基于统计专用字符的维、哈、柯文文种识别研究 | |
| 引用本文： | 买买提依明·哈斯木,吾守尔·斯拉木,维尼拉·木沙江,努尔麦麦提·尤鲁瓦斯.基于统计专用字符的维、哈、柯文文种识别研究[J].中文信息学报,2015,29(2):111-117. |
| 作者姓名： | 买买提依明·哈斯木 吾守尔·斯拉木 维尼拉·木沙江 努尔麦麦提·尤鲁瓦斯 |
| 作者单位： | 1. 新疆大学 信息科学与工程学院,新疆多语种重点实验室,新疆 乌鲁木齐 830046; 2. 和田师范专科学校,计算机科学系,新疆 和田 848000) |
| 基金项目： | 国家“973”重点基础研究计划(2014CB340506);国家自然科学基金(61262063,61363063) |
| 摘 要： | 在Unicode编码方案中维、哈、柯文字符安排在阿拉伯字符区域,三种语言中共享字符比较多,跟阿拉伯字符区域混在一起,没有专用的语言ID。在信息检索和自然语言处理领域对维、哈、柯文的识别、处理带来不便。该文首先分析并总结了维、哈、柯文三种语言中的专用字符、复合字符、某些字符在某种语言中出现形势的独特性等特征,然后在此基础上设计了维、哈、柯文种识别算法。 实验结果表明该文提出的文种识别算法的正确率在文本多于70词时达到96.67%以上。 |
| 关 键 词： | 文种识别 专用字符 复合字符 维文 哈文 柯文 Unicode |
Unique Character Based Statistical Language Identification for Uyghur,Kazak and Kyrgyz |
|
| Maimaitiyiming Hasimu,Wushouer Silamu,Weinila Mushajiang,Nuermaimaiti Youliwasi.Unique Character Based Statistical Language Identification for Uyghur,Kazak and Kyrgyz[J].Journal of Chinese Information Processing,2015,29(2):111-117. | |
| Authors: | Maimaitiyiming Hasimu Wushouer Silamu Weinila Mushajiang Nuermaimaiti Youliwasi |
| Affiliation: | 1. School of Information Science and Engineering, Xinjiang University, Multilingual Information Technology Laboratory of Xinjiang, Urumqi, Xinjiang 830046, China; 2. Department of Computer Science Hotan Teachers College, Hotan, Xinjiang 848000, China |
| Abstract: | In Unicode encoding consortium, Uyghur, Kazak and Kyrgyz characters are arranged in the Arabic characters area and mixed with Arabic characters. Some characters in these languages shares same code without language ID,which brings difficulty in information retrieval and natural language processing. After analyzing the unique characters, compound characters and the special features of some characters in certain language context, this paper designs a language identification algorithm of Uyghur, Kazak and Kyrgyz. The experimental results show that the accuracy achieves 96.67% for texts with 70 words or more. |
| Keywords: | |
| 本文献已被 CNKI 等数据库收录！ | |
| 点击此处可从《中文信息学报》浏览原始摘要信息 | |
| 点击此处可从《中文信息学报》下载全文 | |
