Pass-Join-K:多分段匹配的相似性连接算法  

Pass-Join-K: Similarity Join Method Based on Multi-Match Partition

作　　者：余海洋[1] 林琛[1,2] 陈珂[3] 江弋[1] 邹权[1,2] 

机构地区：[1]厦门大学信息科学与技术学院,福建厦门361005 [2]厦门大学深圳研究院,广东深圳518057 [3]广东石油化工学院计算机科学与技术系,广东茂名525000

出　　处：《计算机科学与探索》2013年第10期924-932,共9页Journal of Frontiers of Computer Science and Technology

基　　金：国家自然科学基金Nos.61102136;61001013;福建省自然科学基金No.2011J05158;深圳市科技创新基础研究No.JCYJ20120618155655087~~

摘　　要：相似性连接是数据清理工作的基本模型，获得了大量数据库工作者的关注。研究了基于编辑距离的相似性连接问题，即在两个字符串集合中寻找编辑距离小于一个阈值的字符串对，并在Pass-Join算法的基础上，提出了一个新的Pass-Join-K算法。Pass-Join—K算法在长短字符串上都有很好的表现。该算法的主要思想是利用Pass-Join算法的划分原理，以多次匹配的方式，达到更加严格地选取候选配对的目的。实验结果显示，Pass-Join-K算法减少了候选对的数量，在实际数据集上相比元算法在运行时间上有2～5倍的提升。Similarity join is the basic model of data cleaning in the database research and has attracted lots of attention from the database community. This paper studies the edit distance based similarity join, which finds similar strings from two large sets of strings whose edit distance is less than a given threshold, and proposes an improved Pass-Join algorithm, named Pass-Join-K. Pass-Join-K is efficient both for short strings and long strings. The main idea of Pass-Join-K is to divide the query string into more parts based on the partition strategy of Pass-Join, and filter the candidate string pairs more strictly by multi-match. The experimental results show that Pass-Join-K can decrease the candidate pairs, and run 2-5 times more quickly than the origin algorithm which outperforms state-of-the-art methods on real datasets.

关 键 词：编辑距离 相似性连接 多次匹配 数据清理 Pass—Join—K算法 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]