基于二进制串的Trie索引树分词词典机制的研究  

Research on Dictionary Mechanism Based on Binary of Trie Index Trees

作　　者：韩莹[1] 丰继林[1] 袁静[2] 王茂发[1] 张艳霞[1] 陈福明[1] 陈新房[1] 潘志安[1] 

机构地区：[1]防灾科技学院灾害信息工程系,北京101601 [2]清华大学计算机科学与技术系,北京100084

出　　处：《计算机与现代化》2013年第1期5-7,共3页Computer and Modernization

摘　　要：分词词典是信息处理系统的一个基本组成部分,其查询效率将直接影响信息处理系统的性能。根据信息在计算机内都是以二进制编码存放的原理,本文把对字符串的处理转化成对二进制串的处理(支持任何语言的字符串),建立基于Trie索引树的分词词典机制。可以根据不同应用系统需求,自动调整二进制串的长度,建立不同的Trie树结构,便于在存储空间和查询效率之间寻找合适的平衡点。这种基于索引的查询速度与词库中词的多少无关,只与词本身的长度有关系;并且公共的前缀索引值随着词汇量的增大而节省大量内存空间。The dictionary mechanism serves as one of the basic components in Chinese word segmentation system. Its performance influences the segmentation speed significantly. Based on binary system, processing of text（ any language string） can be conver- ted into a binary processing, a mechanism of the Tile index tree dictionary can be created. According to different application re- quirements, the dictionary mechanism can automatically adjust the structure of the Tile which can help to find the right balance between storage space and query efficiency. The query speed of this dictionary mechanism is nothing to do with the total word number of dictionary, only influenced by the word length ; Common prefix of the index value with the increase of vocabulary save a lot of memory space.

关 键 词：信息处理 分词 词典机制 Trie索引树 

分 类 号：TP301.6[自动化与计算机技术—计算机系统结构]