无词典抽词的RMMFS和BMMFS方法及其比较研究  

Comparative Study on RMMFS and BMMFS of Chinese Word Extraction without Using Dictionary

作　　者：姜韶华[1] 党延忠[1] 宣照国[1] 

机构地区：[1]大连理工大学系统工程研究所,大连116024

出　　处：《情报学报》2006年第4期499-503,共5页Journal of the China Society for Scientific and Technical Information

基　　金：本文得到国家自然科学基金（项目编号：70271046）资助.

摘　　要：针对作为中文信息处理基础的抽词问题,本文在作者提出的正向串频最大匹配法（MMFS）的基础上,提出了逆向串频最大匹配法（RMMFS）及双向串频最大匹配法（BMMFS）.这两种方法分别采用逆向和双向长串优先与串频统计的思路,并引进规则和支持度指标筛选,不需要词典,不需要事先进行语料库学习,不需要建立字索引,通过串匹配获取中文文本中的汉字共现模式,实时地抽取出包含专业术语及专有名词等未登录词在内的专指语义串、短语和词.实验研究了抽词准确率受规则的影响及随文本大小和词频变化的分布,结果表明BMMFS可以取得更好的抽词效果.To solve the problem of automatic word extraction which is the basis of Chinese information processing, this paper presents two new methods reverse maximum matching and frequency statistics（RMMFS） and bidirectional maximum matching and frequency statistics（ BMMFS）. RMMFS and BMMFS count string frequency by reverse and bidirectional matching longer strings first respectively and obtain co-occurrence patterns of Chinese characters. Refined and filtered by introducing rules and support criteria, special semantic strings, phrases and words, including unknown words like proper nouns and terms, can be extracted in real time without using dictionary, without previous study and without constructing Chinese characters index. The effect of rules and the accuracy distribution of word extraction of RMMFS and BMMFS in different text size and word frequency are studied. Experiments show that BMMFS can get better result.

关 键 词：逆向串频最大匹配 双向串频最大匹配 自动抽词 汉字共现模式 

分 类 号：H164[语言文字—汉语]