一种基于词频歧义消解的通用中文分词法  

A General Method of Chinese Word Segmentation Based on the Resolution of Word Frequency Ambiguity

作　　者：彭琦[1] 朱新华[2] 陈意山[3] 

机构地区：[1]广西师范大学网络中心,广西桂林541004 [2]广西师范大学计算机科学与信息工程学院,广西桂林541004 [3]广西师范大学漓江学院,广西桂林541006

出　　处：《广西师范大学学报（自然科学版）》2016年第1期59-65,共7页Journal of Guangxi Normal University:Natural Science Edition

基　　金：国家自然科学基金资助项目(61363036;61462010)

摘　　要：歧义是在基于词典的分词方法中常见的问题,以往的基于词典的分词方法往往使用双向最大匹配法获得分词结果后,通过使用上下文信息来进行歧义消解,但是对于没有上下文信息的单独语料无法进行歧义消解。本文提出一种通用的基于词频的歧义消解法,该方法是与上下文无关的,能够消解没有上下文信息的语料切分后产生的歧义,扩大了歧义消解的应用范围,简化了歧义消解的处理过程。实验表明:文本方法与传统基于词典的分词算法相比,具有更强的适用性及更高的可用性。Ambiguity is a common problem in dictionary based word segmentation methods.In the past,the word segmentation method based on dictionary often uses the bidirectional maximum matching method to get the result of word segmentation,and then carries out ambiguity resolution by using the context imformation,which cannot be used in the environment without context information.A general disambiguation method based on word frequency is presented in this paper,which is context-free and expands the application range of ambiguity resolution.Experimental results show that compared with the traditional methods of dictionary-based Chinese word segmentation,this method has a stronger applicability and higher availability.

关 键 词：中文分词 词频 歧义消解 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]