基于容器云的分布式深度学习实验平台构建  

Construction of experimental platform for distributed deep learning based on container cloud

作　　者：应毅[1] 刘亚军[2] 任凯 YING Yi;LIU Yajun;REN Kai(College of Computer Science and Technology,Sanjiang University,Nanjing 210012,China;School of Computer Science and Engineering,Southeast University,Nanjing 210096,China;Pujiang Institute,Nanjing Tech University,Nanjing 211200,China)

机构地区：[1]三江学院计算机科学与工程学院,江苏南京210012 [2]东南大学计算机科学与工程学院,江苏南京210096 [3]南京工业大学浦江学院,江苏南京211200

出　　处：《实验技术与管理》2022年第3期147-152,共6页Experimental Technology and Management

基　　金：江苏省现代教育技术研究智慧校园专项课题(2020-R-84352);教育部全国职业教育教师企业实践基地“产教融合”专项课题研究项目“人工智能职业技能等级评价标准研究”;三江学院校级科研重大项目(2019SJKY006)。

摘　　要：该文针对神经网络模型训练过慢、集群部署缺乏弹性问题,采用CPU+GPU混合架构,利用Docker和Kubernetes系统搭建底层容器云基础,以TensorFlow作为上层机器学习框架,构建了分布式深度学习实验平台,并详述了GPU镜像制作、计算节点GPU支持、Kubernetes集群搭建、Tensor Flow集群部署等实验平台建设细节。两个图片识别实验表明,该实验平台具有良好的性能优势和可扩展性,达到了应用部署自动化、硬件资源弹性伸缩、提升模型训练速度的目标,为人工智能实验平台建设提供了有益的思路,并起到推广作用。Aiming at the problems of slow training of neural network model and lack of flexibility in cluster deployment,this article adopts the hybrid architecture of CPU+GPU with Docker and Kubernetes system to construct the underlying container cloud foundation.TensorFlow is also used as the upper machine learning framework to build a distributed deep learning experimental platform.Furthermore,the construction details of the experimental platform such as GPU image making,computing node GPU support,Kubernetes cluster building and TensorFlow cluster deployment are described in detail.Two image recognition experiments show that the experimental platform has good performance advantages and scalability.The goal of automatic application deployment,elastic expansion of hardware resources and improvement of model training speed is achieved,which provides a useful idea and plays a promotional role for the construction of artificial intelligence experimental platform.

关 键 词：深度学习 分布式训练 TensorFlow Kubernetes DOCKER GPU计算 

分 类 号：G482[文化科学—教育学] G642.0[文化科学—教育技术学]