词位标注汉语分词中特征模板定量研究  

Quantitative research on feature templates for word-position-based tagging Chinese word segmentation

作　　者：于江德[1] 王希杰[1] 樊孝忠[2] 

机构地区：[1]安阳师范学院计算机与信息工程学院,河南安阳455002 [2]北京理工大学计算机科学技术学院,北京100081

出　　处：《计算机工程与设计》2012年第3期1239-1244,共6页Computer Engineering and Design

基　　金：高等学校博士学科点专项科研基金项目(20050007023)

摘　　要：基于字的词位标注的方法能极大地提高汉语分词的性能,该方法将汉语分词转化为字的词位标注问题,词位标注汉语分词中特征模板的设定至关重要,为了更加准确地设定特征模板,从多个角度进行了定量分析,并在国际汉语分词评测Bakeoff2005的PKU和MSRA两种语料上进行了封闭测试,得到如下结论:同等条件下,训练出的模型大小与扩展出的特征数成正比;不同的单字特征模板在同一语料中扩展出的特征数基本相同,单字特征模板对分词性能的贡献要比双字特征模板小得多;增加B特征模板之后,训练时间大大增加,模型大小基本不变,对分词性能都是正增长。The performance of Chinese word segmentation is greatly improved by word-position-based approaches in recent years. This approach treats Chinese word segmentation as a word-position tagging problem. Feature template selection is crucial in this method, in order to do better, quantitative analysis on feature templates for word-position-based tagging Chinese word segmentation is implemented. Closed evaluations are performed on PKU and MSRA corpus from the second international Chinese word segmentation Bakeoff-2005, and comparative experiments are performed on different feature templates. Experimental results show the following conclusions： under the same conditions, the trained model size is proportional with the number of features. The number of features by different single character feature template is same. These templates＂ contribution is much smaller than the double-character feature template. With increasing the B feature template, the training time greatly increased, and size of the model remain unchanged, the performance for Chinese word segmentation is positive growth.

关 键 词：汉语分词 词位标注 特征模板 定量分析 条件随机场 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]