中文分词词典结构的研究与改进

汉语分词词典是中文信息处理系统的重要基础,词典算法设计的优劣直接关系着分词的速度和效率。分析了三种典型的分词词典结构,提出了一种具有三级索引的新词典结构,并提出了最大正向匹配的改进型匹配算法,从而降低了匹配过程的时间复杂度。最后通过实验,比较了三种典型词典结构与新词典结构的时间效率。实验结果表明,新词典结构具有更高的词典查询速度和分词速度,可以有效满足中文处理系统的需求。

Research and improvement of Chinese word segmentation dictionary

Chinese word segmentation dictionary is the important base of Chinese information processing system.The arithmetic of the dictionary influences the speed and efficiency of segmentation.This paper analyzes the three typical dictionary structures,and then presents a new dictionary structure using three-level index.On the basis of this new structure,this paper also improves the positive matching method,reduces the complexity of matching process.Then a comparison is given between typical dictionary structures.The result proves that the new dictionary structure is better in many aspects such as the efficiency of retrieving and the speed of Chinese word segmentation and it can fulfill the need of Chinese processing system.