计算存储融合:从高性能计算到大数据  

The Fusion of Computing and Storage: From HPC to Big Data

作　　者：殷进勇[1] 杨阳[1] 徐振朋[1] 姚小城[1] 曾玮妮[1] 

机构地区：[1]江苏自动化研究所,江苏连云港222061

出　　处：《指挥控制与仿真》2015年第3期1-7,共7页Command Control & Simulation

摘　　要：随着军事信息化建设,物联网广泛应用于战场感知、智能控制等军事领域,产生了海量的半结构化、非结构化的数据,受到I/O性能尤其是网络传输、硬盘读写的限制,传统的计算系统难以满足海量数据处理的应用需求。因此,提出了一种计算存储融合方法,通过扩展Linux内核,将集群内所有节点上的内存、处理器等计算存储资源在系统空间映射成一个统一的资源池,实现了单一进程空间和单一内存空间,并在内存空间内建立一个分布式内存文件系统。计算时可将数据完全加载到内存中,计算过程中仅与内存文件系统交互,避免了硬盘读写对系统性能的影响;另外,通过进程迁移,避免了节点之间的大量数据传输。实验结果表明,该方法对数据密集型计算是有效的,能够大幅提升系统的计算性能。With the development of military information, the internet of things is applied widely in some military domains as battlefield perception, intelligent control and so on. As a result, massive semi-structured and no-structure data is produced. Traditional computing system cannot deal with so much data because of the low performance of I/O interfaces especially net- work and hard disks. In this paper, a fusion method of computing and memory is proposed. By extending the Linux kernel, all memory and processors in a cluster are mapped a union resource pool in the system space and form a single memory space and process space. Based on this, distributed RAM file system is build in DRAM. So all data can be loaded into memory and application only read/write RAM file system and reduce hard disk access influence on computing system. On the side, by mi- grating processes between each node, data transportation on the network is avoided. The experimental results show that the method is efficient for data intensive applications and increase computing performance greatly.

关 键 词：计算存储融合 内存计算 进程迁移 分布式内存文件系统 

分 类 号：TP274[自动化与计算机技术—检测技术与自动化装置] TP33[自动化与计算机技术—控制科学与工程]