基于Q学习的蜂窝车联网边缘计算系统PC-5/Uu接口联合卸载策略  

Q-Learning Based Joint PC-5/Uu Offloading Strategy for C-V2X Based Vehicular Edge Computing System

作　　者：冯伟杨 林思雨[1,2] 冯婧涛 李赟 孔繁鹏 艾渤 FENG Wei-yang;LIN Si-yu;FENG Jing-tao;LI Yun;KONG Fan-peng;AI Bo(School of Electronic and Information Engineering,Beijing Jiaotong University,Beijing 100044,China;Collaborative Innovation Center of Railway Traffic Safety,Beijing 100044,China;Information Dispatch Center,China Railway Information Technology Group Co.,Ltd.,Beijing 100089,China;Information Technology Laboratory,China Railway Information Engineering Group Network Technology Research Institute Co.,Ltd.,Beijing 100089,China;Frontiers Science Center for Smart High-Speed Railway System,Beijing 100044,China)

机构地区：[1]北京交通大学电子信息工程学院,北京100044 [2]轨道交通安全协同创新中心,北京100044 [3]中国铁路信息科技集团有限公司信息调度中心,北京100089 [4]中铁信(北京)网络技术研究院有限公司信息技术研究室,北京100089 [5]智慧高铁系统前沿科学中心,北京100044

出　　处：《电子学报》2024年第2期385-395,共11页Acta Electronica Sinica

基　　金：国家重点研发计划(No.2022YFB3207400);国家自然科学基金(No.62221001,No.61971030);中国国家铁路集团有限公司科技研究开发计划资助项目(No.P2021S005)。

摘　　要：智能驾驶等智能交通服务对时延要求高,在车辆本身算力不足的情况下,车辆需要周围车辆和路旁边缘计算单元帮助其一起完成任务的计算处理.本文在既有车联网边缘计算卸载策略基础上,考虑了蜂窝车联网系统5G-NR接口与PC-5接口链路的特征差异,提出了一种基于Q学习的PC-5/Uu接口联合边缘计算卸载策略.在对蜂窝车联网PC-5链路传输成功率进行建模的基础上,推导了PC-5链路的传输速率表征方法.以最小化蜂窝车联网任务处理时延为目标,以任务车辆发射功率与边缘计算车辆的计算能量损耗为约束,构建了系统时延最小化的有约束马尔科夫决策过程.通过拉格朗日方法,将有约束马尔科夫决策过程问题转化为一个等价的极小极大的无约束马尔科夫决策过程,引入Q学习设计卸载策略,进而提出基于Q学习的蜂窝车联网边缘计算系统卸载策略.仿真结果表明,与其他基线方案相比,本文提出的算法可以降低系统时延27.3%以上.Intelligent transportation services,such as smart driving,put forward high requirements for latency.When the vehicle itself has insufficient computing power,the vehicle needs the surrounding vehicles and roadside edge computing units to help it complete the task computation.In this paper,based on the existing vehicular edge computing(VEC)offload⁃ing strategy,considering the differences between the 5G-NR interface and PC-5 interface link of cellular-V2X(C-V2X)sys⁃tem,we propose a Q-Learning based joint PC-5/Uu interface edge computing offloading strategy.The successful transmis⁃sion probability of PC-5 link in C-V2X system is modeled,and then the transmission rate characterization method of PC-5 link is deduced.We formulate a constrained Markov decision process(CMDP)to minimize the system latency,where the objective function is the task processing latency in C-V2X system,and constraints are transmission power at task vehicle and energy consumption of computation at vehicles with edge computing unit.By Lagrangian approach,the CMDP prob⁃lem is transformed into an equivalent min-max non-constrained MDP problem,and Q-Learning is introduced to design the offloading strategy,and then the offloading strategy of C-V2X based VEC system based on Q-Learning is proposed.Simu⁃lation results show that compared with other baseline schemes,the proposed algorithm can significantly improve the system latency performance by at least 27.3%.

关 键 词：蜂窝车联网 边缘计算 有约束马尔科夫过程 计算迁移 Q学习 

分 类 号：TN929.5[电子电信—通信与信息系统]