基于Viterbi改进算法的高棉语分词研究  

Research of Khmer Word Segmentation Based on Improved Viterbi Algorithm

作　　者：蒋艳荣[1] 刘习文[2] 陈耿涛 

机构地区：[1]广东工业大学计算机学院,广州510006 [2]湘潭大学机械工程学院,湖南湘潭411105 [3]广东国笔科技股份有限公司,广州510620

出　　处：《计算机工程》2011年第15期174-176,共3页Computer Engineering

基　　金：广东省自然科学基金资助项目(8151009001000041)

摘　　要：采用最大匹配算法对高棉语进行分词准确率较低,且难以正确识别词库中没有的新词。针对该问题,采用改进的Viterbi算法,利用自动机实现音节切分,通过最优选择及剪枝操作提高分词效率,以统计语言模型对未知新词进行数据平滑,提高识别正确率。实验结果表明,改进的Viterbi算法具有较高的分词效率和准确率。The accuracy of Khmer words segmentation for maximum matching algorithm is relatively low, and it is difficult for this algorithm to recognize words that are not enrolled jn its dictionary. To solve this problem, an improved Viterbi algorithm is proposed. Wherein automation is used for syllable segmentation, optimization selection and pruning methods are used to promote the segmentation efficiency, and the statistical language model is adopted to perform data smooth for unknown words in this approach. Experimental results indicate that the improved Viterbi algorithm has higher accuracy and efficiency.

关 键 词：VITERBI算法 最大匹配算法 分词 高棉语 剪枝 统计语言模型 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]