一种基于无监督学习的交集型歧义处理改进方法  

An Improved Method of Crossing Ambiguities Based on Unsupervised Learning

作　　者：黄鹏[1,2] 张姝[3] 陈玉华[1] 文斌[1] 

机构地区：[1]云南师范大学信息学院,云南昆明650500 [2]云南师范大学民族教育信息化教育部重点实验室,云南昆明650500 [3]云南师范大学档案馆,云南昆明650500

出　　处：《云南师范大学学报（自然科学版）》2015年第6期45-49,共5页Journal of Yunnan Normal University:Natural Sciences Edition

基　　金：国家自然科学基金资助项目(61262071);云南省应用基础研究计划青年资助项目(2013FD015)

摘　　要：歧义处理是中文分词的难点之一,其中交集型歧义占该类问题的主要部分,而无监督学习可以利用互联网上的无标注语料库来处理该问题.文章将比较卡方统计量、t-测试差在歧义处理中的效果,通过对其研究提出一种改善歧义处理性能的方法.实验结果表明,所提出的方法能有效进行分词,并提高了交集型歧义消解的性能.Processing of ambiguities is the key problem in the automatic segmentation of Chinese words. The solution of crossing ambiguities is still an open issue in this problem, and unsupervised learning can make use of the in POS-untagged corpora from internet to deal with the problem. In this article, we compare the chi-square statistic, t-test results in the ambiguity process- ing. Based on analysis of them, some modified methods to further improve its capacity were proposed. The experimental results show that the proposed method can effectively segment Chinese words and improve the performance of crossing ambiguity processing.

关 键 词：分词 无监督学习 交集型歧义 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]