基于密度和半监督学习的数据修复与聚类  

Data repairing and clustering based on density and semi-supervised learning

作　　者：张倩倩[1,2] 李国和[1,2,3] 郑艺峰 ZHANG Qian-qian;LI Guo-he;ZHENG Yi-feng(Beijing Key Lab of Petroleum Data Mining,China University of Petroleum(Beijing),Beijing 102249,China;College of Information Science and Engineering,China University of Petroleum(Beijing),Beijing 102249,China;PanPass Institute of Digital Identification Management and Internet of Things,Beijing 100029,China;Fujian Provincial Key Laboratory of Data Science and Intelligence Applications,Minnan Normal University,Zhangzhou 363000,China;College of Computer Science,Minnan Normal University,Zhangzhou 363000,China)

机构地区：[1]中国石油大学(北京)石油数据挖掘北京市重点实验室,北京102249 [2]中国石油大学(北京)信息科学与工程学院,北京102249 [3]石大兆信数字身份管理与物联网技术研究院,北京100029 [4]闽南师范大学数据科学与智能应用福建省高等学校重点实验室,福建漳州363000 [5]闽南师范大学计算机学院,福建漳州363000

出　　处：《计算机工程与设计》2020年第3期676-681,共6页Computer Engineering and Design

基　　金：国家自然科学基金项目(61701213);油气国家重点专项子课题基金项目(G-5800-08-ZS-WX);中国石油大学(北京)克拉玛依校区科研启动基金项目(RCYJ2016B-03-001);福建省教育厅中青年基金项目(JA15300)。

摘　　要：针对现有数据修复算法需要数据集的完整性依赖等函数、不适用于简单数据集、不能充分利用背景知识等问题,提出一种基于密度和半监督学习的修复和聚类算法。遵循数据修复的最小改变原则,利用样本集自身的密度信息和背景知识形成临时聚类簇;利用成对约束将临时聚类簇进行分割或者合并,形成最终聚类簇,在聚类的同时完成不精确数据的修复。实验结果表明,该算法适用于具有简单模式的样本集,是对现有基于完整性约束数据修复算法的扩展,提高了数据修复正确率和聚类精度。Aiming at the problems of existing data repair algorithms that it requires integrity constraints,it is not suitable for dataset with simple schema and it insufficiently uses background knowledge,a repairing and clustering algorithm based on density and semi-supervised learning was proposed,referring to the principle of minimum change on data repairing.The temporary cluster was formed using the density information and background knowledge of the dataset.The temporary clusters were segmented or merged to form the final cluster using pairwise constraints.The repair of inaccurate data was completed.Experimental results show that the proposed algorithm is suitable for not only dataset with simple schema,but also the existing data repairing algorithms based on integrity constraints,and it improves the accuracy of data repairing and clustering accuracy.

关 键 词：数据质量 数据清理 数据修复 成对约束 密度聚类 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]