基于强化学习的移动视频流业务码率自适应算法研究进展  

Survey on reinforcement learning based adaptive bit rate algorithm for mobile video streaming services

作　　者：杜丽娜[1,2] 卓力 杨硕[1,2] 李嘉锋 张菁[1,2] DU Li’na;ZHUO Li;YANG Shuo;LI Jiafeng;ZHANG Jing(Beijing Key Laboratory of Computational Intelligence and Intelligent System,Beijing University of Technology,Beijing 100124,China;Information Department,Beijing University of Technology,Beijing 100124,China)

机构地区：[1]北京工业大学计算智能与智能系统北京市重点实验室,北京100124 [2]北京工业大学信息学部,北京100124

出　　处：《通信学报》2021年第9期205-217,共13页Journal on Communications

基　　金：国家自然科学基金资助项目(No.61531006);北京市教委-市基金联合资助项目(No.KZ201910005007)。

摘　　要：近几年来,随着HTTP自适应流媒体(HAS)视频数据集和网络轨迹数据集的不断推出,强化学习、深度学习等机器学习方法被不断应用到码率自适应(ABR)算法中,通过交互学习来确定码率控制的最优策略,取得了远超过传统启发式方法的性能。在分析ABR算法研究难点的基础上,重点阐述了基于强化学习(包括深度强化学习)的ABR算法研究进展。此外,总结了代表性的HAS视频数据集和网络轨迹数据集,介绍了算法性能的评价准则,最后探讨了ABR研究目前存在的问题和未来的方向。In recent years,with the continuous release of HTTP adaptive streaming(HAS)video datasets and network trace datasets,the machine learning methods,such as deep learning and reinforcement learning,have been continuously applied to adaptive bit rate(ABR)algorithms,which obtain the optimal strategy of rate control through interactive learning,and achieve superior performance that surpasses the traditional heuristic methods.Based on the analysis of the research difficulties of ABR algorithms,the research advances of ABR algorithms based on reinforcement learning(including deep reinforcement learning)was investigated.Furthermore,several representative HAS video datasets and network trace datasets were summarized,the evaluation metrics of the performance were depicted.Finally,the existing problems and the future tendency of ABR research were discussed.

关 键 词：强化学习 码率自适应算法 用户质量体验 深度学习 深度强化学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]