一种基于分布式平台Hadoop的矩阵相乘算法  

Algorithm of Distributed Matrix Multiplication Based on Hadoop

作　　者：冯健[1] 倪明[1] 赵建波[1] 

机构地区：[1]中国电子科技集团公司第三十二研究所,上海200233

出　　处：《计算机系统应用》2013年第12期149-154,共6页Computer Systems & Applications

基　　金：国家"863"计划基金重点项目(2009AA012201);上海市科委科技攻关基金重大项目(08dz501600)

摘　　要：为了解决单节点上大矩阵相乘运算量过大的问题,提出了一种基于分布式平台Hadoop的矩阵相乘算法.算法采用了字节文件作为输入以及最优化的分片方式,去掉了不必要的Reduce过程,极大地减少了输入数据量,拥有简洁的算法流程和可拓展性.实验结果表明该算法很好地解决了矩阵相乘运算量过大的问题,当输入数据量接近集群最佳负载量的情况下取得了很好的加速比.In order to release the burden of large matrix multiplication on a single node, a distributed matrix multiplication algorithm based on Hadoop is proposed. The algorithm uses the input file with binary format, applys optimal split, removes the unnecessary Reduce phase. The algorithm can greatly reduce the amount of input data, has simple algorithm flow and good scalability. Experiment results demonstrate that the algorithm greatly reduces the computation of matrix multiplication and achieves good speedup when the amount of input data is around the optimum loading of the cluster.

关 键 词：HADOOP平台 分布式矩阵乘法 输入格式 分片方式 

分 类 号：TP302[自动化与计算机技术—计算机系统结构]