中文新词识别技术综述  

Survey of Chinese New Words Identification

作　　者：张海军[1,2] 史树敏[3] 朱朝勇[1] 黄河燕[3] 

机构地区：[1]中国科学技术大学计算机科学与技术学院,合肥230027 [2]新疆师范大学计算机系,乌鲁木齐830054 [3]中国科学院计算机语言信息工程研究中心,北京100097

出　　处：《计算机科学》2010年第3期6-10,16,共6页Computer Science

基　　金：国家自然科学基金项目(60672149);国家863计划重点项目(2006AA010109)资助

摘　　要：新词识别是中文信息处理领域的关键技术。新词识别主要包括候选字串的提取过滤和词性猜测两项任务。中文没有特定符号标志词边界,因此任何相邻字符都有成词的可能性,这给新词提取过滤带来了很大困难;由于没有先验知识和统计数据,新词词性猜测一直是中文词性标注的技术瓶颈。详细分析了中文新词识别技术的研究现状,重点讨论了候选新词提取和词性猜测的研究方法与存在的主要问题,最后对新词识别研究方向进行了展望。New Words Identification （NWI） is a key technology in the field of Chinese information processing. NWI mainly includes two tasks;one is new words candidate extracting and filtering, the other is new words POS guessing.Since there is no specific symbol to mark word boundary for Chinese words,any adjacent characters are possible to compose a word, which brings a lot of obstacles for NWI. Moreover, because the prior knowledge and statistical data are not available, new words POS guessing has become the technological bottleneck of Chinese tagging. The status of the field for Chinese NWI was analyzed in detail, and the research techniques and existing problems for new words candidates extrading and new words POS guessing were discussed emphatically. In the end, the paper presented the prospects of the study for Chinese NWI.

关 键 词：新词识别 未登录词 候选字串 训练语料 词性猜测 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]