面向大数据复杂应用的虚拟集群动态部署模型  

Virtual cluster dynamic deployment model for complex applications in large-scale data processing

作　　者：王瑾 曹云鹏[1,2] 王海峰 Wang Jin;Cao Yunpeng;Wang Haifeng(School of Information Science&Engineering,Linyi University,Linyi Shandong 276002,China;Linda Institute,Shandong Provincial Key Laboratory of Network Based Intelligent Computing,Linyi Shandong 276002,China)

机构地区：[1]临沂大学信息科学与工程学院,山东临沂276002 [2]山东省网络重点实验室临沂大学研究所,山东临沂276002

出　　处：《计算机应用研究》2020年第6期1760-1764,共5页Application Research of Computers

基　　金：山东省自然科学基金面上项目(ZR2017MF050);山东省高等学校科学技术计划项目(J17KA049);山东省重点研发项目(2018GGX101005,2017CXGC0701,2016GGX109001)。

摘　　要：针对计算负载的时变性和复杂性导致虚拟集群的资源利用率不高的问题,为提高虚拟集群资源的全局利用率,采用弹性资源管理策略来吸收多种计算模式混杂时的资源需求突变。在Docker容器技术的支持下提出一个根据作业需求变化的动态部署模型。该模型根据资源的动态需求变化,实时调整虚拟集群的计算形态,具体包括计算节点的类型及规模。该模型不仅实现用户作业执行环境的动态定制,而且达到错峰计算的目的。仿真实验表明,该模型使得虚拟节点CPU利用率提升5. 3%,并且优化了计算作业的执行效率。该动态部署模型适合应用到数据中心或大规模集群中,能够有效提高计算资源的利用率。To deal with the low resource utilization of virtual cluster due to the time-varying and complexity of workloads,this paper used the flexible resource management strategy to improve the global resource utilization of virtual cluster and absorb the mutation of resource demand when multiple computing modes were mixed. This paper proposed a novel dynamic deployment model based on Docker. According to the dynamic demand of workloads,this model could change the computing form of virtual cluster in real time. Its computing form included the node type and cluster size. This dynamic deployment model not only realized the dynamic customization of job execution environment,but also achieved the purpose of virtual cluster peak calculation. Simulation experiments show that this model can improve the virtual node CPU utilization by 5. 3% and improve the task execution efficiency. This model is suitable for data centers or large-scale clusters to improve the utilization of computing resources.

关 键 词：虚拟集群 动态部署 大数据复杂应用 Docker容器 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]