基于拟牛顿法的深度强化学习在车联网边缘计算中的研究  

Research on deep reinforcement learning in Internet of vehicles edge computing based on Quasi-Newton method

作　　者：章坚武[1] 芦泽韬 章谦骅 詹明 ZHANG Jianwu;LU Zetao;ZHANG Qianhua;ZHAN Ming(School of Communication Engineering,Hangzhou Dianzi University,Hangzhou 310018,China;Research Center for Space Computing System,Zhejiang Lab,Hangzhou 311121,China;College of Information Science and Electronic Engineering,Zhejiang University,Hangzhou 310027,China;College of Electronic and Information Engineering,Taizhou University,Taizhou 318000,China)

机构地区：[1]杭州电子科技大学通信工程学院,浙江杭州310018 [2]之江实验室天基计算研究中心,浙江杭州311121 [3]浙江大学信息与电子工程学院,浙江杭州310027 [4]台州学院电子与信息工程学院,浙江台州318000

出　　处：《通信学报》2024年第5期90-100,共11页Journal on Communications

基　　金：浙江省自然科学基金重点项目(No.LZ23F010001)。

摘　　要：为了解决车联网中由于多任务和资源限制导致的任务卸载决策不理想的问题,提出了拟牛顿法的深度强化学习双阶段在线卸载(QNRLO)算法。该算法首先引入批归一化技术优化深度神经网络的训练过程,随后采用拟牛顿法进行优化,有效逼近最优解。通过此双阶段优化,算法显著提升了在多任务和动态无线信道条件下的性能,提高了计算效率。通过引入拉格朗日算子和重构的对偶函数,将非凸优化问题转化为对偶函数的凸优化问题,确保算法的全局最优性。此外,算法考虑了车联网模型中的系统传输时间分配,增强了模型的实用性。与现有算法相比,所提算法显著提高了任务卸载的收敛性和稳定性,并能有效处理车联网中的任务卸载问题,具有较高的实用性和可靠性。To address the issues of ineffective task offloading decisions caused by multitasking and resource constraints in vehicular networks,the Quasi-Newton method deep reinforcement learning dual-phase online offloading(QNRLO)al‐gorithm was proposed.The algorithm was designed by initially incorporating batch normalization techniques to optimize the training process of deep neural networks.Subsequently,optimization was performed using the Quasi-Newton method to effectively approximate the optimal solution.Through this dual-stage optimization,performance was significantly en‐hanced under conditions of multitasking and dynamic wireless channels,improving computational efficiency.By intro‐ducing Lagrange multipliers and a reconstructed dual function,the non-convex optimization problem was transformed into a convex optimization problem of the dual function,ensuring the global optimality of the algorithm.Additionally,system transmission time allocation in the vehicular network model was considered,enhancing the practicality of the al‐gorithm.Compared to existing algorithms,the proposed algorithm improves the convergence and stability of task offloading significantly,addresses task offloading issues in vehicular networks effectively,and offers high practicality and reliability.

关 键 词：车联网 任务卸载 深度强化学习 拟牛顿法 

分 类 号：TN92[电子电信—通信与信息系统]