汉语自动分词词典机制的实验研究  

An Experimental Study on Dictionary Mechanism for Chinese Word Segmentation

作　　者：孙茂松[1] 左正平[1] 黄昌宁[1] 

机构地区：[1]清华大学计算机科学与技术系,北京100084

出　　处：《中文信息学报》2000年第1期1-6,共6页Journal of Chinese Information Processing

基　　金：国家自然科学基金资助!(合同号:69433010)

摘　　要：分词词典是汉语自动分词系统的一个基本组成部分。其查询速度直接影响到分词系统的处理速度。本文设计并通过实验考察了三种典型的分词词典机制:整词二分、TRIE 索引树及逐字二分,着重比较了它们的时间、空间效率。实验显示:基于逐字二分的分词词典机制简洁、高效,较好地满足了实用型汉语自动分词系统的需要。The dictionary mechanism serves as one of the basic components in Chinese word segmentation systems.Its performance influences the segmentation speed significantly.In this paper,we design and implement three typical dictionary mechanisms,i.e.binary seek by word,TRIE indexing tree and binary seek by characters,from word segmentation point of view,and compare their space and time complexity experimentally.It can be seen that the binary seek by characters model is the most appropriate one being capable of fulfilling the need for speed of practical Chinese word segmenters to the maximum extent.

关 键 词：中文信息处理 汉语自动分词 分词词典机制 

分 类 号：H085.5[语言文字—语言学] TP391[自动化与计算机技术—计算机应用技术]