一种基于逐层扫描的频繁字串快速提取算法  

An Algorithm of Fast Frequent String Extracting Based on Level-wise Scan

作　　者：张宇萌[1] 刘传汉[1] 

机构地区：[1]上海交通大学计算机科学与工程系

出　　处：《计算机科学》2008年第5期127-130,共4页Computer Science

摘　　要：串频统计是一种简便有效的抽取未登录词方法。本文提出了一种快速的频繁字串提取和计频方法,通过逐层扫描快速发现频繁字串,修正字串有效出现频次,最后抽取平均互信息量达到阚值的字串。实验结果显示该方法有效可行。String frequency statistics is a simple and effective method of extraction unlisted word. This paper presents an effective algorithm of extracting frequent strings. It uses a level-wise scan for finding rapidly frequent strings and modifies the valid frequency that string appears in text. Finally, those high-frequent strings that reach the threshold of average mutual information are extracted. Experimental results show that the method is effective and feasible.

关 键 词：频繁字串 中文抽词 逐层扫描 互信息 

分 类 号：TP311.13[自动化与计算机技术—计算机软件与理论] TP311[自动化与计算机技术—计算机科学与技术]