基于MongoDB的BESIII分布式计算记账系统的研究与实现  

RESEARCH AND APPLICATION OF MONGODB-BASED BESIII DISTRIBUTED COMPUTING ACCOUNTING SYSTEM

作　　者：张刚[1] 孙涌[1] 邓子艳[2] 李卫东[2] 张晓梅[2] 

机构地区：[1]苏州大学计算机科学与技术学院,江苏苏州215006 [2]中国科学院高能物理研究所,北京100049

出　　处：《计算机应用与软件》2015年第9期62-67,共6页Computer Applications and Software

基　　金：国家自然科学基金项目(11205180,11375221,11179020,11121092)

摘　　要：北京谱仪III(BESIII)高能物理实验要对海量的数据进行处理与分析。为了应对海量数据对计算资源提出的挑战,BESIII实验基于分布式架构软件DIRAC构建了统一的分布式计算平台。针对实验具体需求,对BESIII分布式计算的记账系统进行研究,以MongoDB作为系统底层存储数据库,采用复合片键以及副本集策略,提高系统可扩展性与可用性;提出基于动态时间窗口的数据压缩算法,实现了统计查询结果的实时响应;基于Map/Reduce、Matplotlib提高分析速度以及实现分析结果的多样化展示。目前该系统已被用于计算平台信息的收集和分析。测试表明基于MongoDB的记账系统表现出了良好的性能,能够满足实验的需要。The high energy physics experiments carried out by Beijing Spectrometer III （BESIII） need to process and analysing massive data. In order to tackle the challenges raised by the massive data on computing resources, BESIII experiments constructed the uniform distributed computing platform based on the distributed architecture software DIRAC. In light of specific requirements of experiments, we studied the accounting system of BESIII distributed computation, used MongoDB as the underlying storage database of the system, and employed compound shard key and replica sets strategy to improve the scalability and availability of the system. Furthermore, we presented the dynamic time window-based data compression method and realised real-time responses on the statistic query results; based on Matplotlib and Map/Reduce we improved analysis speed and implemented the diversified display of analysis results. Currently, the system has been applied to the data collection and analysis on the computing platform. Test showed that the accounting system based on MongoDB was in good performance and could meet the requirements of the experiment.

关 键 词：记账系统 MONGODB 分布式计算 数据压缩 高能物理 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术]