高维相关性缺失数据的分块填补算法研究  

Research on Block Imputation Algorithm for High Dimensional Correlation Missing Data

作　　者：杨杰 杨虎[1] 王鲁滨[1] 金鑫[1] 郭华 于亮亮 YANG Jie;YANG Hu;WANG Lubin;JIN Xin;GUO Hua;YU Liangliang(School of Information, Central University of Finance and Economics, Beijing 100081, China;Jingzhou Power Supply Company ICT Branch of State Grid Corporation, Jingzhou, Hubei 434000, China;Liaoning Power Supply Company ICT Branch of State Grid Corporation, Shenyang 110000, China)

机构地区：[1]中央财经大学信息学院 [2]国网荆州供电公司信通分公司 [3]国网辽宁省电力有限公司信息通信分公司

出　　处：《计算机科学与探索》2017年第10期1557-1569,共13页Journal of Frontiers of Computer Science and Technology

基　　金：中央财经大学青年教师发展基金No.QJJ1510;国家电网科技部项目No.SGTYHT/14-JS-188~~

摘　　要：研究了高维相关性缺失数据的填补方法,提出了分块填补算法。该算法核心思想是:在填补数据的过程中会考虑变量之间的相互关系,仅利用与待填补数据有相关性的数据进行填补,从而降低不相关数据对缺失数据填补的影响,提高数据填补的准确度。同时,该算法能够并行处理缺失数据,从而提高数据填补效率,对于高维缺失数据的填补有重要意义。为了对分块情况未知的缺失数据进行分块,提出了基于k-means聚类的分块算法。大量的仿真实验和基于真实数据集的实验表明,对于相关性数据,分块填补算法能够有效地利用相关信息进行填补,从而提高数据填补准确度。This paper studies the method of filling the high dimensional correlation missing data,and proposes a new imputation algorithm based on data block.The key idea of the algorithm is to consider the correlation between variables when filling missing data,and only use the data correlated with the missing data to fill,thereby reducing imputation effects of the missing data caused by the irrelevant data,and improving the accuracy of data imputation.At the same time,the proposed imputation algorithm can be implemented in a parallel way,so that it performs efficiently to fill the high dimensional missing data.In order to divide the missing data with unknown information about blocks into several blocks,this paper proposes a block algorithm based on k-means clustering.Simulation research and application show that the proposed imputation algorithm is more effective and accurate to handle the missing for the correlation high dimensional data with considering variables'block relationship than others with not.

关 键 词：高维相关性数据 缺失数据 分块填补算法 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]