基于稀疏表示和特征加权的大数据挖掘方法的研究  

Study on Big Data Mining Method Based on Sparse Representation and Feature Weighting

作　　者：蔡柳萍[1] 解辉[2] 张福泉[3] 张龙飞[3] CAI Liu-ping;XIE Hui;ZHANG Fu-quan;ZHANG Long-fei(School of Computer Science&Engineering,Tianhe College of Guangdong Polytechnic Normal University,Guangzhou 510540,China;Department of Computer Sciences and Technology,Tsinghua University,Beijing 100084,China;School of Software,Beijing Institute of Technology,Beijing 100081,China)

机构地区：[1]广东技术师范学院天河学院计算机科学与工程学院,广州510540 [2]清华大学计算机科学与技术系,北京100084 [3]北京理工大学软件学院,北京100081

出　　处：《计算机科学》2018年第11期256-260,共5页Computer Science

基　　金：文化部国家科技支撑计划项目(2012BAH38F00);广东省本科高校应用型人才培养课程建设项目:能力培养导向的计算机类应用型课程建设(2017SZ03);广东省科技计划项目:基于医药电商大数据的服务系统研发(2016A010101029);广东技术师范学院天河学院计算机科学与技术重点学科建设项目(Xjt201702)资助

摘　　要：为了提高大数据挖掘的效率及准确度,文中将稀疏表示和特征加权运用于大数据处理过程中。首先,采用求解线性方程稀疏解的方式对大数据进行特征分类,在稀疏解的求解过程中利用向量的范数将此过程转化为最优化目标函数的求解。在完成特征分类后进行特征提取以降低数据维度,最后充分结合数据在类中的分布情况进行有效加权来实现大数据挖掘。实验结果表明,相比于常见的特征提取和特征加权算法,提出的算法在查全率和查准率方面均呈现出明显优势。In order to improve the efficiency and accuracy of big data mining,this paper applied the sparse representation and feature weighting into big data processing.At first,the features of big data are classified by solving the sparse mode of linear equation.In the process of solving the sparse solution,a vector norm is utilized to transform this process into the process of solving the optimization objective function.After feature classification,feature extraction is executed to reduce the dimensionality of data.Finally,the distribution of data in the class is combined sufficiently to conduct weighting effectively,thus realizing data mining.The experimental results suggest that the proposed algorithm is supe-rior to the common feature extraction and feature weighting algorithms in the terms of recall and precision.

关 键 词：大数据 数据挖掘 特征加权 特征提取 稀疏表示 

分 类 号：TP301[自动化与计算机技术—计算机系统结构]