基于HBase的大数据架构下负载平衡技术  

Load Balancing Technology Under Big Data Architecture Based on HBase

作　　者：雷鸣 姜罕盛 武国良[1] 赵玉娟[1] 梁健 LEI Ming;JIANG Han-sheng;WU Guo-liang;ZHAO Yu-juan;LIANG Jian(Tianjin Meteorological Information Center, Tianjin 300074, China)

机构地区：[1]天津市气象局信息中心,天津300074

出　　处：《计算机与现代化》2021年第6期91-95,共5页Computer and Modernization

基　　金：国家自然科学基金资助项目(41575156);天津市气象局科研项目(201914ybxm12)。

摘　　要：随着气象数据规模和种类的不断增长,气象数据已经逐渐进入海量服务阶段,而基于大数据背景提供更敏捷的数据服务已经成为业务发展的迫切需求。本文针对气象中的半/非结构化数据,提出基于HBase系统的负载平衡算法和策略。在实际测试对比中发现,系统可以满足200多万个格点,100个并发的场景,查询速度在2 s以内,与未曾增加负载平衡算法相比,系统数据响应速度提升了42.69倍,能够有效地满足实际业务需要。With the continuous growth of the scale and type of meteorological data,meteorological data has gradually entered the stage of massive service,and meanwhile providing more agile data services based on the background of big data has become an urgent demand for business development.In this paper,a load balancing algorithm and strategy based on HBase system is proposed for semi/unstructured meteorological data.In the actual test and comparison,it is found that the system can meet more than 2 million grid points and 100 concurrent scenarios,and the query speed is within 2 s.Compared with the load balancing algorithm which is not added load balance,the system data response speed is improved by 42.69 times,which can effectively meet the actual business needs.

关 键 词：负载平衡 HBASE 分布式存储 大数据 HDFS 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]