基于MapReduce模型的非结构化数据分布式存储方法  

Unstructured Data Distributed Storage Method Based on MapReduce Model

作　　者：梁雪青 杜舒明 LIANG Xueqing;DU Shuming(Guangzhou Power Supply Bureau,Guangzhou 510000,China)

机构地区：[1]广东电网广州供电局,广东广州510000

出　　处：《微型电脑应用》2022年第7期106-109,共4页Microcomputer Applications

摘　　要：为了解决传统数据存储方法存在的内存消耗过大、读取效率低等多种问题,提出了基于MapReduce模型的非结构化数据分布式存储方法。设计了非结构化数据垂直分片与水平分片方案,并采用聚类算法对分片处理后的非结构化数据进行聚簇处理,在此基础上构建用于数据分布式存储的MapReduce模型,得到非结构化数据分布式存储结果。实验测试结果表明,与传统非结构化数据存储方法相比,研究方法的内存消耗更小、写入量更大、读取效率更高、访问频率动态调节性能好,由此证明该方法的存储性能更优越,实际应用效果更好。In order to solve problems existing in traditional data storage methods,such as excessive memory consumption and low reading efficiency,a distributed unstructured data storage method based on MapReduce model is proposed.The vertical and horizontal sharding schemes of unstructured data are designed,and the unstructured data after sharding areclustered by clustering algorithm.On this basis,a MapReduce model for distributed data storage is built,and the distributed storage results of unstructured data are obtained.The experimental results show that compared with the traditional unstructured data storage methods,the research method has smaller memory consumption,larger writing volume,higher reading efficiency and better performance in dynamic regulation of access frequency.

关 键 词：人工智能 非结构化数据 分布式存储 存储消耗 

分 类 号：TP392[自动化与计算机技术—计算机应用技术]