行列混合存储的数据压缩策略研究  

Research on Data Compression Strategy of Row-column Mixed Storage

作　　者：魏玲[1] 郭新朋[1] 

机构地区：[1]哈尔滨理工大学管理学院,哈尔滨150040

出　　处：《小型微型计算机系统》2017年第6期1267-1272,共6页Journal of Chinese Computer Systems

基　　金：国家自然科学基金项目(71272191)资助;黑龙江省哲学社会科学研究规划项目(16GLD02)资助

摘　　要：大数据技术不断深入发展,对于数据压缩技术的要求也在不断提高.为此,构建叠加式行列混合存储结构,提出动态编码选择数据压缩策略.首先,依据数据类型,选取对应处理效果最佳的编码方式;然后,设计基于行列存储的页式存储结构对数据属性缓冲区中的数据进行处理,确定可压缩的属性数据并存储于属性块;最后设计并实现动态编码选择数据压缩算法,对属性块中的数据进行压缩处理,计算数据压缩比.应用大规模数据集SSB中的数据表进行数据压缩处理,实验结果验证了基于行列混合存储的动态编码选择数据压缩策略在数据压缩比、数据压缩时间方面性能优于单一行、列存储下的数据压缩方式,提高了压缩效率.Big data technology continues to develop, the requirements for data compression technology is also constantly improving. Therefor, to construct the superposition type mixed storage structure, and propose the dynamic coding selection data compression strategy. According to the data type,selects the best way to deal with the corresponding treatment effect. Then,designs the page storage structure to deal with data attribute buffer＇s data, determines the attribute datas can be compressed and stores to attribute blocks. At last, designs and achieves the dynamic coding selection data compression algorithm for compressing the data of attribute blocks, and computes the data compression ratio. It＇s important to applicate the large-scale data set SSB＇s data tables to compress the data. And the experimental results verify the dynamic code selection data compression strategy which is based on row-column mixed storage in data compression ratio, time has better performance than the single row or column storage, and improves the compression efficiency.

关 键 词：行列混合存储 页式存储 数据压缩 数据压缩比 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]