具有概念联想功能的特定领域分词词典的自动构建  

Automatic Creation of Special Domain Word-segmentation Dictionary with Concept Association

作　　者：张彦[1] 邵志清[1] 

机构地区：[1]华东理工大学计算机科学与工程系,上海200237

出　　处：《计算机工程》2004年第20期148-150,共3页Computer Engineering

基　　金：上海市青年科技启明星计划(99QD14038)

摘　　要：提出了一种基于PAT树型结构的高频字串提取的改进算法,并以此用来获得特定领域网页中的未登录词集合,利用基于语义距离的概念相似度计算公式来获得任一概念的相关概念,从而给出了用于特定领域搜索引擎的语义词典完整的自动构建方法。将生成的语义词典用于搜索引擎FlyingSender中。实验结果证明新的词典比原有的手工构建的词典分词效果要理想得多,而且提供了概念联想的功能。An improved algorithm based on PAT tree structure to extract high frequency strings appearing in the doc is presented. Through it the paper gains a set of words of specific domain not listed in the general dictionary, computes the similarity for each two concepts to obtain associative concepts based on semantic distance. Finally, it concludes a complete automatic method to construct a thesaurus of specific domain used by search engine. Experimental results prove new thesaurus with concept association functions makes better performance than the general one.

关 键 词：语义词典 未登录词 PAT树 语义距离 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]