基于Hash结构词典的逆向回溯中文分词技术研究  

Reverse backtracking research of Chinese segmentation based on dictionary of Hash structure

作　　者：梁桢[1] 李禹生[1] 

机构地区：[1]武汉工业学院计算机与信息工程系,湖北武汉430023

出　　处：《计算机工程与设计》2010年第23期5158-5160,F0003,共4页Computer Engineering and Design

基　　金：湖北省教育厅科研基金项目(D200618003)

摘　　要：为了提高现有的中文分词效率,提出了基于Hash结构词典的逆向回溯中文分词方法。针对首字Hash结构词典的不足,设计了能够记录词长的Hash结构尾字词典,然后对逆向最大匹配分词算法进行了分析,为了解决其存在的中文分词歧义问题,设计出一种逆向回溯最大匹配算法,该改进算法采用的回溯机制能够有效消除分词中可能存在的一些歧义问题。实验结果表明,该方法实现了提高中文分词速度并减少交集型歧义字符串切分错误的设计目标。To improve the efficiency of the existing Chinese word segmentation,a design method of the reverse backtracking algorithm based on dictionary with Hash Structure is presented.Firstly,for the insufficiency of first word dictionary with hash structure,a last word dictionary based Hash structure is designed to record words length.Then,the reverse maximum matching segmentation algorithm is analyzed.In order to solve the existence problems of ambiguity in Chinese word segmentation,the reverse backtracking algorithm is designed.The possible ambiguities of the segmentation are eliminated by the backtracking mechanism of improved algorithm.Experiments show that the proposed method improves the speed and reduces the ambiguities of the segmentation.

关 键 词：中文分词 哈希结构 尾字词典 逆向最大匹配算法 分词歧义 逆向回溯算法 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]