面向查询式实体解析的多属性数据索引技术  

Multi-attribute Data Indexing for Query Based Entity Resolution

作　　者：孙琛琛 申德荣 肖迎元[1] 李玉坤[1] SUN Chen-Chen;SHEN De-Rong;XIAO Ying-Yuan;LI Yu-Kun(Key Laboratory of Computer Vision and System of Ministry of Education(Tianjin University of Technology),Tianjin 300384,China;School of Computer Science and Engineering,Northeastern University,Shenyang 110169,China)

机构地区：[1]计算机视觉与系统省部共建教育部重点实验室(天津理工大学),天津300384 [2]东北大学计算机科学与工程学院,辽宁沈阳110169

出　　处：《软件学报》2022年第6期2331-2347,共17页Journal of Software

基　　金：国家自然科学基金(62002262,61672142,61602103,62072086,62072084);国家重点研发计划(2018YFB1003404)。

摘　　要：实体解析是数据集成的关键方面,也是大数据分析与挖掘的必要预处理步骤.大数据时代,随着查询驱动的数据应用需求的不断增长,查询式实体解析成为热点问题.为了提升查询-解析效率,研究了面向实体缓存的多属性数据索引技术.涉及两个核心问题:(1)如何设计多属性数据索引?设计了基于R-树的多属性索引结构.为了满足实体缓存在线生成需求,提出了基于空间聚类的在线索引构建方法.提出了基于“过滤-验证”的多维查询方法,利用多属性索引有效地过滤掉不可能命中的记录,然后采用相似性函数或距离函数逐一验证候选记录.(2)如何将不同的字符串属性插入到树形索引中?解决思路是,将字符串映射到数值空间.针对Jaccard相似性和编辑相似性,提出了基于q-gram的映射方法,并提出了基于向量降维的优化和基于z-order的优化,实现高质量的“字符串→数值”映射.最后,在两个数据集上进行实验评估,验证多属性索引的有效性,并测试其各个方面.Entity resolution is a key aspect of data integration, and also is a necessary preprocessing step of big data analytics and mining.In big data era, more and more query-driven data analytics applications come out, and query-based entity resolution becomes a hot topic.This work studies multi-attribute data indexing technology for entity cache in order to promote query-resolution efficiency. There are two core problems. One is how to design the multi-attributeindex. An R-tree based multi-attributeindex is designed. Entity cache is produced online, so an online index construction method is proposed based on spatial clustering. A filter-verify based multi-dimensional query method is proposed. It filters impossible records by the multi-attributeindex, and then verifies each candidate record with similarity functions or distance functions. The other ishow to insert different string attributes into the tree index. The basic solution is mapping strings into integer spaces. For Jaccard similarity and edit similarity, a q-gram based mapping method is proposed, and is improved by vector dimension reduction and z-order, which achieves high mapping qualities. Finally, the proposed hybrid index is experimentally evaluated on two datasets. Its effectiveness is validated, and moreover, different aspects of the multi-attribute index are also tested.

关 键 词：实体解析 多属性数据索引 查询式 数据集成 数据预处理 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]