基于Hadoop的学习行为数据云存储平台的设计与实现  

Design and Implementation of Cloud Storage Platform for Learning Behavior Data based on Hadoop

作　　者：蔡春花 黄思远 高继梅 CAI Chunhua;HUANG Siyuan;GAO Jimei(Huanghe Jiaotong University,Jiaozuo 454000,China)

机构地区：[1]黄河交通学院,河南焦作454000

出　　处：《软件工程》2022年第10期50-53,49,共5页Software Engineering

基　　金：河南省科技攻关项目“MOOC的大数据学习行为与成效分析技术研究”(222102320218);黄河交通学院校级课程教学资源库建设项目“Hadooop分布式基础架构”(HHJTXY-2021kczyk091);黄河交通学院校级一流课程建设项目“Hadooop分布式基础架构”(HHJTXY-2022ylkc49)。

摘　　要：文中设计一个用于存储的平台,通过虚拟化技术实现了服务器,规划了平台的存储集群规模及服务;对存储集群架构、请求处理系统、数据库进行了设计;使用Spring Boot和Thymeleaf实现了前台用户功能模块,包括用户注册、用户登录、文件上传及文件相关操作、用户关注和用户分享等功能;最后对HBase库亿级大数据查询性能和Hadoop集群输入/输出(Input/Output,I/O)性能进行了测试。测试结果表明查询性能可以做到百毫秒级查询,写性能平均I/O速率为91.73 Mb/s,读性能平均I/O速率为348.56 Mb/s,能够满足用户需求。This paper proposes to design a storage platform which realizes server through virtualization technology and plans the storage cluster scale and services of the platform.The storage cluster architecture,request processing system,and database are also designed.Spring Boot and Thymeleaf are used to achieve function modules of front-end users,such as user registration and login,file upload and file-related operations,user attention and sharing.Finally,the billion-level big data query performance of HBase library and the Input/Output(I/O)performance of Hadoop cluster are tested.Test results show that the query performance can reach hundred milliseconds-level query,the average I/O rate of write performance is 91.73 Mb/s,and the average I/O rate of read performance is 348.56 Mb/s.The proposed platform can meet users’need.

关 键 词：云存储 学习行为 HADOOP集群 查询性能 I/O性能 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]