基于Hash结构的机械统计分词系统研究  

A Kind of Machine-Statistics System Based on Hash Structure for Chinese Word Segmentation

作　　者：费洪晓[1] 胡海苗[1] 巩燕玲[1] 

机构地区：[1]中南大学信息科学与工程学院,湖南410075

出　　处：《计算机工程与应用》2006年第5期159-161,共3页Computer Engineering and Applications

基　　金：国家自然科学基金资助项目(编号:60173041);湖南省自然科学基金资助项目(编号:02JJY2094)

摘　　要：在综合比较了常用的机械分词方法和统计分词方法基础上,论文设计并实现了一种基于Hash结构的机械统计分词系统。系统突破了传统的机械分词方法和统计分词方法,结合了两者的优势,并对其做了一系列的改进。从对测试结果的分析可以看出,系统的分词速度达到了每秒一万两千个汉字以上,且具有较强的未登陆词识别能力。On the basis of comprehensive comparison of the mechanical Chinese word segmentation and statistical Chinese word segmentation which are often used,a kind of machine-statistics system based on Hash structure for Chinese word segmentation is put forward and carried out.This system breaks through the traditional mechanical Chinese word segmentation method and statistical Chinese word segmentation method and combines the advantages of them.On the other hand,this thesis makes a series of improvement on these two methods as well.The analysis of the testing results shows that the segmentation speed of this system can reach more than 12,000 Chinese characters pre second, further more,this system also has great capacity for finding the new words which not exit in the dictionary.

关 键 词：中文分词 机械分词 统计分词 Hash结构 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]