基于深度强化学习的无人机辅助弹性视频多播机制  

Deep Reinforcement Learning Based UAV Assisted SVC Video Multicast

作　　者：成昭炜 沈航 汪悦[1] 王敏 白光伟 CHENG Zhao-wei;SHEN Hang;WANG Yue;WANG Min;BAI Guang-wei(College of Computer Science and Technology,Nanjing Tech University,Nanjing 211816,China;State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210093,China)

机构地区：[1]南京工业大学计算机科学与技术学院,南京211816 [2]南京大学计算机软件新技术国家重点实验室,南京210093

出　　处：《计算机科学》2021年第9期271-277,共7页Computer Science

基　　金：国家自然科学基金项目(61502230);江苏省自然科学基金项目(BK20201357);江苏省“六大人才高峰”高层次人才资助项目(RJFW-020);计算机软件新技术国家重点实验室资助项目(KFKT2017B21);江苏省研究生科研与实践创新计划项目(KYCX20_1079,SJCX20_0351);国家教育部2019年第二批产学合作协同育人项目(201902182003)。

摘　　要：文中提出了一个异构网络下无人机基站辅助的弹性视频多播机制。结合SVC编码,将无人机动态部署和资源分配问题联合考虑,目的是最大化用户整体的视频质量。考虑到宏基站覆盖范围内用户的移动会使网络拓扑结构发生改变,传统的启发式算法难以应对用户移动的复杂性。对此,采用基于深度强化学习的DDPG算法训练神经网络来决策无人机的最佳部署位置和带宽资源分配比重。在模型收敛后,学习代理可以在较短的时间内找到最优的无人机部署和带宽分配策略。仿真结果表明,所提方案达到了预期目标并且优于现有的基于Q-learning的方案。In this paper,a flexible video multicast mechanism assisted by the UAV base station is proposed.In combination with SVC encoding,the dynamic deployment and resource allocation of UAV are considered jointly in order to maximize the overall number of enhancement layers received by users.The traditional heuristic algorithm is difficult to deal with the complexity of user movement,considering that the user movement within the range of macro station will change the network topology.To this end,the DDPG algorithm based on deep reinforcement learning is used to train the neural network to decide the optimal location and bandwidth allocation proportion of UAV.After the model converges,the learning agent can find the optimal UAV deployment and bandwidth allocation strategy in a short time.The simulation results show that the proposed scheme achieves the expected goal and is superior to the existing scheme based on Q-learning.

关 键 词：可伸缩视频编码 多播 深度强化学习 无人机 移动互联网 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]