自动答疑系统中文分词模块的设计与实现  

Design and Research of Chinese word Segmentation Module in automatic answering system

作　　者：马新意 王剑辉 MA Xin-yi;WANG Jian-hui

机构地区：[1]沈阳师范大学数学与系统科学学院,辽宁沈阳110034

出　　处：《信息技术与信息化》2019年第1期19-22,25,共5页Information Technology and Informatization

基　　金：辽宁省教育科学"十二五"规划2014年度课题(课程批准号:JG14DB379)成果

摘　　要：本文对国内外自动答疑系统的研究现状进行了分析,对常用的分词词典机制和分词算法进行了理论研究,深入讨论了分词过程中常出现的歧义问题,提出了基于双字哈希索引的词典机制,并将改进的最大匹配算法与串频统计相结合,提高了中文分词的准确性,对自动答疑系统中的中文分词模块进行了设计与实现,通过实验证明该分词系统具有良好的切分精度和实用性。In this paper,the current research situation of automatic question answering system at home and abroad is analyzed,the commonly used word segmentation dictionary mechanism and word segmentation algorithm are theoretically studied,and the ambiguity problems that often occur in the process of word segmentation are discussed in depth.A dictionary mechanism based on two-word hash index is proposed,and the improved maximum matching algorithm is combined with string frequency statistics to improve the accuracy of Chinese word segmentation.The Chinese word segmentation module in the automatic answering system is designed and implemented.Experiments show that the segmentation system has good segmentation accuracy and practicability.

关 键 词：中文分词算法 分词词典机制 最大匹配算法 交集型歧义 

分 类 号：G434[文化科学—教育学] TP391.1[文化科学—教育技术学]