基于MongoDB海量地形小文件入库优化方法  

Storage Optimization Method for Massive Terrain Small Files Based on MongoDB Database

作　　者：许伟平[1] 吴晨[1] 朱庆[1,2] 张叶廷[1] 谢潇[1] 熊庆[1] 何峰[1] 

机构地区：[1]武汉大学测绘遥感信息工程国家重点实验室,湖北武汉430079 [2]西南交通大学地球科学与环境工程学院,四川成都611756

出　　处：《地理信息世界》2014年第6期72-76,共5页Geomatics World

基　　金：国家科技支撑计划项目(2012BAH35B02)资助;国家自然科学基金项目(41471332)资助;青年科学基金项目(41301439)资助;绿洲生态教育部重点实验室开放课题项目(XJDX0201-2013-01)资助

摘　　要：针对关系型数据库无法满足海量地形小文件存储的高效性和扩展性,非关系型数据库通过最终一致性保证了高效的入库效率和可扩展架构,但存在入库安全隐患的现状。本文提出了基于非关系型分布式数据库Mongo DB的海量地形小文件入库优化方法。该方法充分发挥MongoDB内存文件映射方式带来的性能优势,通过客户端周期性阻塞实现内外存同步,以保证海量地形小文件并发写入的安全性;将入库的文件信息批量写入日志,支持工程级别的数据库回滚,以保持数据库的正确性。通过客户端周期阻塞与日志批量入库的协同,极大地保证了地形高效入库的安全性。Relational databases cannot satisfy the efficiency and scalability of storing massive terrain small files. Non-relational databases guarantee the efficient storage and scalable architecture through eventual consistency, but it confronts security risks when storing. This paper proposes the storage optimization method for massive terrain small file based on non-relational distributed database MongoDB. This method makes full use of the performance advantage of memory file mapping in MongoDB, periodically synchronizing internal and external memory by client blocking which greatly ensures the security of concurrent writing. In addition, writing the information of terrain small files into log can support the database rollback in project level, maintaining the correctness of the database. Through the collaboration of client periodical blocking and log batch writing, this method greatly ensures the safety and high efficiency of storing terrain small files.

关 键 词：地形 并发写入 入库安全 MONGODB 

分 类 号：P2[天文地球—测绘科学与技术] TP392[自动化与计算机技术—计算机应用技术]