基于统计方法的Web新词分词方法研究  

Study on New Words of Web Based on Statistical Word Segmentation

作　　者：张敏[1] 王春红[1] 

机构地区：[1]运城学院计算机科学与技术系,山西运城044000

出　　处：《计算机工程与科学》2010年第5期133-135,共3页Computer Engineering & Science

基　　金：山西省高等学校科技开发项目(20091150);运城学院项目(JC-2009009)

摘　　要：本文对信息处理技术中各种分词方法进行了研究,针对目前分词方法无法识别网络中不断出现的新词,设计了一种新的基于统计的分词方法。该方法避开现有的分词方法中的复杂语法规则,无需词典的支持,很好地解决了新词不断出现的问题,而且分词速度快,具有重要的理论和实用价值。This paper analyzes the various segmentation methods in the information processing technology. In view of the current segmentation methods in the network which do not recognize the new emerging words, we design a new sub-word method based on statistics. This method avoids complex grammar and rules, needs no enormous support from dictionaries, and resolves the problems brought by the new words. So we conclude that this method has better exactness and is very pragmatic and powerful in practical operations.

关 键 词：WEB 统计分词 词典 特征提取 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]