基于Web的重复属性自动识别方法  

Automatic Web-based duplicate attribute resolution method

作　　者：胡嘉琪[1] 陈群[1] 刘海龙[1] 杜晶[1] 徐曜[1] 李战怀[1] 

机构地区：[1]西北工业大学计算机学院,西安710129

出　　处：《计算机工程与应用》2015年第9期125-128,共4页Computer Engineering and Applications

基　　金：国家"973"重点基础发展规划基金(No.2012CB316203);西北工业大学研究生种子基金(No.Z2013125;No.Z2013126)

摘　　要：在建立数据仓库的过程中,需要从多个数据源导入数据。这些数据存在大量相似重复记录,严重影响了数据利用率和决策质量。因此,相似重复记录的检测已经成为数据仓库等领域的热点研究问题,而重复属性的识别是完成相似重复记录检测的关键。提出一种高效的基于Web的重复属性自动识别算法,该算法使用搜索引擎返回的摘要和URL信息计算属性相似度,并使用查询探针提高查询准确度。实验结果表明该算法有较高的查全率。It needs to import data from different sources when building a data warehouse, which results in the increase of approximately duplicated records, adversely affecting the data utilization and the quality of making decisions. Therefore, detecting approximately duplicated records has become an intensive research subject. Duplicate attribute resolution is the key of approximately duplicated records detecting. This paper proposes an efficient, Web-based algorithm to automatically recognize duplicated attributes. This novel algorithm uses snippets and URL information returned by search engine to calcu-late attribute similarity, and further improves recognition precision by inserting query probes. The results show that the algo-rithm improves recall.

关 键 词：重复属性识别 WEB搜索 摘要 查询探针 

分 类 号：TP31[自动化与计算机技术—计算机软件与理论]