一种适宜于子空间聚类的离群点检测算法  

An Outlier Detection Algorithm for Subspace Clustering

作　　者：杨维永 何军[2] 郑生军 张旭东[4] 

机构地区：[1]南京南瑞集团公司,江苏南京210003 [2]南京信息工程大学电子与信息工程学院,江苏南京210044 [3]北京国电通网络技术有限公司,北京100070 [4]国网浙江省电力公司信息通信分公司,浙江杭州310007

出　　处：《计算机与现代化》2015年第12期39-42,共4页Computer and Modernization

基　　金：国家自然科学基金资助项目(61203273);国家电网公司科技项目(524681140009)

摘　　要：大数据处理面临数据信息缺失、信息杂乱、数据污染等诸多挑战,而离群点的检测与剔除是大数据分析中一个重要的预处理过程。本文针对数据分析中的子空间聚类问题,在数据受到离群点污染的情况下,基于离群点稀疏性的先验知识,对经典的k-subspace子空间聚类算法融入1范数正则化,并采用随机梯度下降优化方法进行子空间聚类,以克服大数据处理计算量大和内存需求高的困难。通过数值仿真,本文方法能够保证在数据受到严重离群点污染的条件下,可以精确检测并剔除离群异常数据,从而获得准确的子空间聚类结果。There are several challenging difficulties in modern big data analytics, such as missing data, unstructured data, and outlier corruption, etc. The foremost important preprocess is outlier detection and removal. In this paper, for tackling the popular subspaee clustering problem in data analyties, we consider the more challenging scenario in which the data set is corrupted by sparse outliers. Based on the sparsity assumption, the classic k-subspace algorithm is adapted to incorporate the l1 norm regularization to alleviate outlier side-effect. In order to overcome the huge requirements of computation and memory in big data, the modified k-subspace clustering algorithm exploits stochastic gradient descent （SGD） for fast computation and memory efficiency. Simulation experiments show that even the data set is heavily corrupted by outliers the proposed approach can guarantee to accu- rately detect and remove outliers, and furthermore achieves the accurate subspaee clustering results.

关 键 词：大数据处理 离群点检测 子空间聚类 

分 类 号：TP312[自动化与计算机技术—计算机软件与理论]