基于Hash结构的逆向最大匹配分词算法的改进  

Improvement on reverse directional maximum matching method based on hash structure for Chinese word segmentation

作　　者：丁振国[1] 张卓[1] 黎靖[1] 

机构地区：[1]西安电子科技大学计算机学院,陕西西安710071

出　　处：《计算机工程与设计》2008年第12期3208-3211,3265,共5页Computer Engineering and Design

基　　金：国家863高技术研究发展计划基金项目(2004AA1Z2520);军队网络互联与信息安全策略研究基金项目(2006QB1069)

摘　　要：分析中文的语义,首先要对句子进行分词。中文分词是中文信息处理中最重要的预处理,分词的速度和精度直接影响信息处理的结果。对传统的分词词典和算法进行了改进,提出了基于Hash结构的分词词典机制,并给出了一种改进的逆向最大匹配分词算法(RMM)。该算法在重点考虑切分速度的同时兼顾了切分精度,在很大程度上消除了传统的最大匹配算法可能产生的歧义。实验结果表明,该分词算法在运行效率和结果的准确性方法有了很大的提高。To analyse the Chinese semantic phrases, one must divide the sentences into words. Chinese segmentation is the most important part of Chinese information process. The speed and accuracy of segmentation influence the results of information processing. Traditional dictionary mechanisms and word segmentation methods are improved. Meanwhile, a new dictionary mechanism is provided based on hash structure, and an improved reverse directional maximum match method （RMM） is put forward. This method emphasized particularly on the speed of segmentation and the accuracy of segmentation, and it largely dispelled some ambiguities that may be produced by traditional maximum matching method. The experiment indicates that the segmentation method is improved obviously on running efficiency and veracity of the results.

关 键 词：中文分词 哈希结构 逆向最大匹配算法 分词词典 消除歧义 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]