基于分布式非负矩阵分解的大规模主题社区挖掘  

Large-scale topic community mining based on distributed nonnegative matrix factorization

作　　者：贺超波[1,2] 汤庸[2] 杨阿祧[3,2] 赵淦森[2] 刘海[2] 黄昌勤[2] 

机构地区：[1]仲恺农业工程学院信息科学与技术学院,广州510225 [2]华南师范大学计算机学院,广州510631 [3]贵州师范大学数学与计算机科学学院,贵阳550001

出　　处：《中国科学：信息科学》2016年第6期714-728,共15页Scientia Sinica(Informationis)

基　　金：国家自然科学基金(批准号:61370178,61370229);国家高技术研究发展计划(批准号:2013AA01A212);国家科技支撑计划项目(批准号:2012BAH27F05,2014BAH28F02);广东省自然科学基金(批准号:S2012030006242,2015A030310509);广东省科技计划项目(批准号:2015A020209178);广东省高性能计算重点实验室开放课题(批准号:TH1527);广州市云计算安全与测评技术重点实验室开放基金(批准号:GZCSKL-1407)资助项目

摘　　要：复杂网络的主题社区挖掘具有重要的应用价值,但现有方法可扩展性差,无法高效挖掘大规模复杂网络的主题社区.针对该问题,提出一种基于分布式非负矩阵分解的主题社区挖掘方法:TCMDNMF(topic community mining based on distributed nonnegative matrix factorization),该方法基于非负矩阵联合分解模型,可以有效统一集成节点链接和内容信息挖掘主题社区.通过采用梯度下降方法对主题社区挖掘模型进行了优化求解,并引入L1范数作为稀疏性正则项以及基于Map Reduce分布式计算框架提高了关键算法的计算效率.实验结果表明,TCMDNMF不仅可以有效挖掘主题社区,而且具有高度可扩展性,可以有效解决大规模复杂网络主题社区挖掘带来的大数据量计算问题.Mining topic community in complex networks is of great applicable value. However, most of the existing methods cannot effectively mine topic community in large-scale complex networks because of their weak scalabilities. To rectify this problem, we propose a method called TCMDNMF that is based on the joint nonnegative matrix factorization model. The proposed method can effectively integrate node link and content information to mine topic community. We adopt the gradient descent method as the optimized solution to the topic community mining model. Further, to improve the computing efficiency of TCMDNMF, we use L1 norm as the sparsity regularization term and implement the key algorithms based on the Map Reduce distributed computing framework. The results of extensive experiments conducted demonstrate that our method is effective and is highly scalable. Furthermore, it very effectively solves the problem of processing large volumes of data brought by topic community mining in large-scale complex networks.

关 键 词：复杂网络 主题社区挖掘 非负矩阵分解 MAPREDUCE 梯度下降 

分 类 号：O157.5[理学—数学]