面向算力网络的跨域数据管理方法  

Cross-domain data management for computing power networks

作　　者：鲁蔚征 戴奇志 张策 LU Weizheng;DAI Qizhi;ZHANG Ce(Office of Research Infrastructure,Renmin University of China,Beijing 100872,China;Lianjingzhineng Technology(Shanghai)Co.,Ltd.,Shanghai 200051,China;Network and Computing Center,Huazhong University of Science&Technology,Wuhan 430074,China)

机构地区：[1]中国人民大学大型科学仪器共享平台,北京100083 [2]联旌智能科技(上海)有限公司,上海200051 [3]华中科技大学网络与计算中心,湖北武汉430074

出　　处：《大数据》2024年第2期94-108,共15页Big Data Research

基　　金：国家重点研发计划资助项目(No.2020YFB1710004)。

摘　　要：跨域算力网络希望整合多个算力中心的计算和数据资源,但现有的方案对跨域文件和数据管理关注不够。提出了一种轻量级的跨域算力网络数据管理方案:通过文件系统协议转换,接入远程算力中心的并行文件系统存储资源;算力中心内部的存储资源作为一种补充,应对高IOPS应用;通过容器绑定技术,将远程存储挂载并绑定到指定目录。基于该方案的原型系统已经在高校校级计算平台部署运行。实测数据和用户体验显示,该方案能够满足常见高性能计算应用需求。Cross-domain computing power networks wish to integrate computational and data resources from multiple computing centers,but existing methods do not pay enough attention to cross-domain file and data management.In this paper,a lightweight data access scheme for cross-domain computing power networks was proposed:(1)accessing parallel file system storage resources of remote computing centers through file system protocol conversion;(2)local caching as a supplement to cope with high IOPS applications;and(3)mounting remote or local storage to specified directories through container binding technology.The prototype system based on this scheme had been deployed on highperformance computing centers in multiple universities.The measured data and user experience showed that the scheme in this paper could meet the requirements of common high-performance computing applications.

关 键 词：算力网络 并行文件系统 数据管理 异构存储资源 

分 类 号：TP316[自动化与计算机技术—计算机软件与理论]