基于Hadoop的大矩阵乘法处理方法  

Approach of large matrix multiplication based on Hadoop

作　　者：孙远帅[1] 陈垚[1] 官新均 林琛[1,2] 

机构地区：[1]厦门大学信息科学与技术学院,福建厦门361005 [2]厦门大学深圳研究院,广东深圳518000

出　　处：《计算机应用》2013年第12期3339-3344,3358,共7页journal of Computer Applications

基　　金：国家自然科学基金资助项目(61102136;61001013);福建省自然科学基金资助项目(2011J05158;2010J01351);深圳市战略性新兴产业发展专项资金资助项目(JCYJ20120618155655087)

摘　　要：目前的矩阵乘法算法无法处理大规模和超大规模的矩阵,而随着MapReduce编程框架的提出,并行处理矩阵乘法成为解决大矩阵运算的主要手段。总结了矩阵乘法在MapReduce编程模型上的并行实现方法,并提出了实现高性能大矩阵乘法的策略———折中单个工作节点的计算量和需要网络传输的数据量。实验证明,并行实现算法在大矩阵上明显优于传统的单机算法,而且随着集群中节点数目的增多,并行算法会表现出更好的性能。Large and very large matrix cannot be dealt by current matrix multiplication algorithms. With the development of MapReduce progranlming frame, parallel programs have become the main approaches for matrix computing. The matrix multiplication algorithms based on MapReduce were summarized, and an improved strategy for large matrix was proposed, which had a tradeoff in the data volume between the computation on single work node and the network transmission. The experimental results prove that the parallel algorithms outperform the traditional ones on the large matrix, and the performance will imorove with the increase of the c｜lJ^t~r~

关 键 词：大矩阵 矩阵乘法 矩阵运算 MAPREDUCE HADOOP 并行计算 海量数据 

分 类 号：TP391[自动化与计算机技术—计算机应用技术] TP311[自动化与计算机技术—计算机科学与技术]