基于Hadoop的海量MP3文件存储架构  

Hadoop-based storage architecture for mass MP3 files

作　　者：赵晓永[1] 杨扬[1] 孙莉莉[1] 陈宇[1] 

机构地区：[1]北京科技大学计算机与通信工程学院,北京100083

出　　处：《计算机应用》2012年第6期1724-1726,共3页journal of Computer Applications

基　　金：国家自然科学基金资助项目(61070182;60873192)

摘　　要：MP3作为数字音乐时代的事实标准,数量已相当巨大,用户的访问需求也与日俱增,有效存储和管理海量的MP3文件,为用户提供良好的体验,越来越引起人们的关注。Hadoop的出现,提供了一种新的思路。但由于Hadoop自身并不适合海量小文件处理,提出一种基于Hadoop的海量MP3文件存储架构,充分利用MP3文件自身包含的丰富描述信息,通过预处理模块的归类算法,将小文件归并到Sequence File中,同时引入高效的索引机制,很好地解决了小文件过多时NameNode内存瓶颈问题。实验证明,该架构能达到良好的效果。MP3 as the de facto standard for digital music,the number of files is quite large and user access requirements rapidly grow up.How to effectively store and manage vast amounts of MP3 files to provide good user experience has become a matter of bigger concern.The emergence of Hadoop provides us with new ideas.However,because Hadoop itself is not suitable for handling massive small files,this paper presented a Hadoop-based storage architecture for massive MP3 files,fullly using the MP3 file＇s rich metadata.The classification algorithm by pre-processing module would merged small files into sequence file,and the introduction of efficient indexing mechanism served as a good solution to the problem of small files.The experimental results show that the approach can achieve a better performance.

关 键 词：HADOOP MP3 小文件 富元数据 存储架构 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]