一种基于深度学习的异常数据清洗算法  

An Outlier Cleaning Algorithm Based on Deep Learning

作　　者：匡俊搴 赵畅 杨柳 王海峰[5] 钱骅[1,2,3] KUANG Junqian;ZHAO Chang;YANG Liu;WANG Haifeng;QIAN Hua(Shanghai Advanced Research Institute,Chinese Academy of Sciences,Shanghai 201210,China;School of Information Science and Technology,ShanghaiTech University,Shanghai 201210,China;University of Chinese Academy of Sciences,Beijing 100049,China;School of Microelectronics,University of Chinese Academy of Sciences,Beijing 100049,China;Shanghai Institute of Microsystem and Information Technology,Chinese Academy of Sciences,Shanghai 200050,China)

机构地区：[1]中国科学院上海高等研究院,上海201210 [2]上海科技大学信息科学与技术学院,上海201210 [3]中国科学院大学,北京100049 [4]中国科学院大学微电子学院,北京100049 [5]中国科学院上海微系统与信息技术研究所,上海200050

出　　处：《电子与信息学报》2022年第2期507-513,共7页Journal of Electronics & Information Technology

基　　金：国家自然科学基金(61971286);国家重点研究发展计划(2020YFB2205603);上海市科学技术委员会科技创新行动计划(19DZ1204300)。

摘　　要：在物联网(IoT)中采用合适的异常数据清洗算法能极大地提升数据质量。许多研究人员采用统计学方法或分类聚类等方法对时-空相关数据进行清洗。但这些方法需要额外的先验知识,会给汇聚节点带来额外的计算开销。该文根据低秩-稀疏矩阵分解模型,提出一种基于深度神经网络的快速异常数据清洗算法,来解决物联网中时-空相关数据的清洗问题。结合感知数据的时-空相关性和异常值的稀疏性,将异常数据清洗问题转换为优化问题,并采用迭代阈值收缩算法(ISTA)求解该优化问题,再将ISTA算法展开成一个固定长度的深度神经网络。实际数据集的实验结果表明,该方法能够自动更新阈值,比传统的ISTA算法收敛速度更快,精度更高。The use of appropriate abnormal data cleaning algorithms in the Internet of Things(IoT)can greatly improve data quality.Statistical methods or clustering methods are utilized to clean anomalies in Spatiotemporal data.However,these methods require additional prior knowledge,which will incur additional computational overhead for the sink node.In this paper,in line with the low-rank sparse matrix decomposition model,a fast anomaly cleaning algorithm based on a deep neural network is proposed to solve the Spatiotemporal data cleaning problem in IoT.Both the Spatio-temporal correlation of sensing data and the abnormal values'sparsity are considered in an optimization problem.The Iterative Shrinkage-Thresholding Algorithm(ISTA)is used to solve it.Then the ISTA is unfolded into a fixed-length deep neural network.The real-world dataset’s experimental results show that the proposed method can automatically update the thresholds faster and more accurately than the traditional ISTA.

关 键 词：物联网 异常数据清洗 迭代阈值收缩算法 展开 深度神经网络 

分 类 号：TN915[电子电信—通信与信息系统] TP181[电子电信—信息与通信工程]