异种语料融合方法:基于统计的中文词法分析应用  

A Method of Merging Corpora in Different Annotation Standards:An Application on Statistics Chinese Lexical Analysis

作　　者：孟凡东[1] 徐金安[2] 姜文斌[1] 刘群[1] 

机构地区：[1]中国科学院计算技术研究所智能信息处理重点实验室,北京100190 [2]北京交通大学计算机与信息技术学院,北京100044

出　　处：《中文信息学报》2012年第2期3-7,12,共6页Journal of Chinese Information Processing

基　　金：国家自然科学基金资助项目(60873167;60736014);中央高校基本科研业务费专项资金项目(2009JBM027)

摘　　要：基于统计的中文词法分析往往依赖大规模标注语料,语料的规模和质量直接影响词法分析系统的性能。高覆盖率、高质量的语料资源非常有限,而且适用于不同领域的语料往往具有不同的分词和词性标注标准,难以直接混合使用,从而导致既有资源未能充分利用,分词精度下降等问题。针对该问题,该文提出了简单有效的异种语料的自动融合方法,并通过实验验证了提案方法的有效性、较强的实用性以及对多种语料融合的可扩展性。Large scale manually annotated corpora are usually used in research on statistical Chinese lexical analysis.The scale and quality of corpora affect the performance of statistical lexical analysis directly.Corpora in high quality and high rate of coverage are very valuable but limited,and it is very difficult to combine corpora of different domains directly since they are different in segmentation and part of speech（POS） tagging standards.These problems make it difficult to utilize existing resources and prevent the performance improvment in Chinese lexical analysis.To address this issue,this paper presents a simple but effective strategy to optimize the performance and domain adaptability of Chinese lexical analysis by merging different domains＇ corpora automatically.Our experiments verify the validity,the stronger practicability,and the scalability to multiple corpora of the proposed method.

关 键 词：词法分析 语料融合 领域适应 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]