Research on performance optimization of virtual data space across WAN  

作　　者：Jiantong HUO Zhisheng HUO Limin XIAO Zhenxue HE 

机构地区：[1]State Key Laboratory of Software Development Environment,Beihang University,Beijing 100191,China [2]School of Computer Science and Engineering,Beihang University,Beijing 100191,China [3]High Performance Computing Center,Beihang University,Beijing 100191,China [4]Hebei Key Laboratory of Agricultural Big Data,Hebei Agricultural University,Baoding 071001,China

出　　处：《Frontiers of Computer Science》2024年第6期167-187,共21页计算机科学前沿（英文版）

基　　金：the National Natural Science Foundation of China(Grant Nos.62104014,62272026);the National Laboratory of Software Development Environment(No.SKLSDE-2022ZX-07);the Hebei Youth Talents Support Project(No.BJ2019008);the Natural Science Foundation of Hebei Province(No.F2020204003)。

摘　　要：For the high-performance computing in a WAN environment,the geographical locations of national supercomputing centers are scattered and the network topology is complex,so it is difficult to form a unified view of resources.To aggregate the widely dispersed storage resources of national supercomputing centers in China,we have previously proposed a global virtual data space named GVDS in the project of“High Performance Computing Virtual Data Space”,a part of the National Key Research and Development Program of China.The GVDS enables large-scale applications of the high-performance computing to run efficiently across WAN.However,the applications running on the GVDS are often data-intensive,requiring large amounts of data from multiple supercomputing centers across WANs.In this regard,the GVDS suffers from performance bottlenecks in data migration and access across WANs.To solve the above-mentioned problem,this paper proposes a performance optimization framework of GVDS including the multitask-oriented data migration method and the request access-aware IO proxy resource allocation strategy.In a WAN environment,the framework proposed in this paper can make an efficient migration decision based on the amount of migrated data and the number of multiple data sources,guaranteeing lower average migration latency when multiple data migration tasks are running in parallel.In addition,it can ensure that the thread resource of the IO proxy node is fairly allocated among different types of requests(the IO proxy is a module of GVDS),so as to improve the application’s performance across WANs.The experimental results show that the framework can effectively reduce the average data access delay of GVDS while improving the performance of the application greatly.

关 键 词：storage aggregation across WANs large-scale applications GVDS data migration allocation of IO proxy resource 

分 类 号：TP393.2[自动化与计算机技术—计算机应用技术] TP333[自动化与计算机技术—计算机科学与技术]