基于群体智能算法的大数据分布式存储方法  

Distributed Storage Method of Big Data Based on Swarm Intelligence Algorithm

作　　者：胡媛媛 江春然 甘杜芬[2] HU Yuan-yuan;JIANG Chun-ran;GAN Du-fen(Department of Artificial Intlligence and Big Data,Hebei Polytechnic Institute,Shijiazhuang Hebei 050000,China;School of Computer Engineering,Guilin University of Electronic Technology,Beihai Guangxi 536000,China)

机构地区：[1]河北工程技术学院人工智能与大数据学院,河北石家庄050000 [2]桂林电子科技大学计算机工程学院,广西北海536000

出　　处：《计算机仿真》2023年第11期447-451,共5页Computer Simulation

基　　金：2020年河北省人力资源社会保障研究课题(JRS-2020-1030)。

摘　　要：采用目前方法对大数据进行存储时,无法有效分配数据中存在的冗余数据,导致方法存在带宽占用率高、负载均衡度低、节点剩余能量低和数据存储量低的问题。提出基于群体智能算法的大数据分布式存储方法,利用一致性树分布存储方法完成大数据的分布存储,通过群体智能算法选择存储节点,对存储节点和大数据进行映射处理,将数据映射到对应的存储节点中,并采用遗传算法对大数据中存在的冗余数据进行分配,完成大数据的分布式存储。实验结果表明,所提方法的带宽占用率低、负载均衡度高、节点剩余能量高、数据存储量高。Currently,some methods fail to effectively allocate redundant data.Therefore,a method of distributed storage for big data based on a swarm intelligence algorithm was proposed.This method used the consistency tree dis-tributed storage to complete the distributed storage of big data,and selected storage nodes by the swarm intelligence algorithm.And then,the storage nodes and big data were mapped to storage nodes correspondingly.Moreover,a genet-ic algorithm was adopted to allocate the redundant data existing in the big data.Finally,the distributed storage was completed.Experimental results show that the proposed method can achieve low bandwidth occupancy,high load bal-ance,high node residual energy and high data storage capacity.

关 键 词：群体智能算法 一致性树分布存储 数据映射 大数据分布式存储 

分 类 号：TP311.12[自动化与计算机技术—计算机软件与理论]