三位一体字标注的汉语词法分析  

A Unified Character-Based Tagging Approach to Chinese Lexical Analysis

作　　者：于江德[1] 胡顺义[1] 余正涛[2] 

机构地区：[1]安阳师范学院计算机与信息工程学院,河南安阳455000 [2]昆明理工大学信息工程与自动化学院,云南昆明650051

出　　处：《中文信息学报》2015年第6期1-7,共7页Journal of Chinese Information Processing

基　　金：国家自然科学基金(60863011);河南省基础与前沿技术研究计划项目(112300410182);河南省教育厅科学技术研究重点项目(14A520077)

摘　　要：针对汉语词法分析中分词、词性标注、命名实体识别三项子任务分步处理时多类信息难以整合利用,且错误向上传递放大的不足,该文提出一种三位一体字标注的汉语词法分析方法,该方法将汉语词法分析过程看作字序列的标注过程,将每个字的词位、词性、命名实体三类信息融合到该字的标记中,采用最大熵模型经过一次标注实现汉语词法分析的三项任务。并在Bakeoff2007的PKU语料上进行了封闭测试,通过对该方法和传统分步处理的分词、词性标注、命名实体识别的性能进行大量对比实验,结果表明,三位一体字标注方法的分词、词性标注、命名实体识别的性能都有不同程度的提升,汉语分词的F值达到了96.4%,词性标注的标注精度达到了95.3%,命名实体识别的F值达到了90.3%,这说明三位一体字标注的汉语词法分析性能更优。To integrate multi-information without error accumulation in the pipeline approach, a unified characterbased tagging approach is proposed for Chinese lexical analysis, including word segmentation, part-of-speech tag- ging and named entity recognition. Treating Chinese lexical analysis as a character sequence tagging problem, each character tagging could be integrated with three kinds of information that is word-position, part-of-speech and named entity. After the tagging process, the maximum entropy model is applied to complete the three subtasks. The closed evaluation is performed on PKU corpus from Bakeoff2007, and the results show a F-score of 96.4% on word segmentation, 95.3 % on POS tagging and 90.3 % on named entity recognition.

关 键 词：汉语词法分析 最大熵模型 三位一体 字标注 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]