基于深度学习的大规模数据库重复记录删除研究  

Research on large scale database′s duplicate record deletion based on deep learning

作　　者：职晓晓 ZHI Xiaoxiao(Henan University Minsheng College,Kaifeng 475000,China)

机构地区：[1]河南大学民生学院,河南开封475000

出　　处：《现代电子技术》2021年第5期114-116,共3页Modern Electronics Technique

基　　金：2018年度河南省科技攻关项目:基于视频图像处理的交通流量数据采集和检测技术研究(82102410064)。

摘　　要：重复记录直接影响数据库管理的效率,当前重复记录删除的正确率低、误删率高,为了提高重复记录删除的正确率,满足重复记录管理的要求,提出基于深度学习的大规模数据库重复记录删除方法。首先对已有数据库重复记录删除方法进行分析,找到引起数据库重复记录删除不理想的因素;然后采用深度学习算法对数据库重复记录数据进行学习,建立数据库重复记录的分类器,根据分类结果进行数据库重复记录删除操作;最后通过仿真实验分析数据库重复记录删除效果。结果表明,深度学习算法可以对数据库重复记录进行准确分类和识别,获得较高的删除正确率,误删率明显小于其他数据库重复记录删除方法,研究结果具有一定的理论和实际价值。Duplicate records will directly affect the efficiency of database management.The accuracy of the current duplicate record deletion is low and the rate of false deletion is high.In order to improve the accuracy of duplicate record deletion and meet the requirements of duplicate record management,a method of large scale database′s duplicate record deletion based on deep learning is proposed.The available methods of deleting the duplicate records in existing databases are analyzed to find out the factors that cause unsatisfactory deletion.And then,the deep learning algorithm is adopted to learn the data of the duplicate records in databases,so as to establish the classifier for duplicate records in databases.The database duplicate records are deleted according to the results of classification.The simulation experiments were carried out to analyze the deletion effect of duplicate records in databases.The results show that the deep learning algorithm can accurately classify and identify duplicate records in databases,get higher deletion accuracy,and its rate of false deletion is significantly less than that of the other methods.Therefore,the research results have a certain theoretical and practical value.

关 键 词：重复记录删除 大规模数据库 数据库管理 分类器建立 深度学习算法 效果分析 

分 类 号：TN911.1-34[电子电信—通信与信息系统] TM614[电子电信—信息与通信工程]