强化学习在自适应视频码率控制算法中的应用  

Auto-tuning Video ABR Algorithms Based on Q-learning

作　　者：肖强 白光伟 沈航 XIAO Qiang;BAI Guang-wei;SHEN Hang(College of Computer Science and Technology.Nanjing Tech University.Nanjing 211816,China)

机构地区：[1]南京工业大学计算机科学与技术学院,南京211816

出　　处：《小型微型计算机系统》2020年第2期252-258,共7页Journal of Chinese Computer Systems

基　　金：国家自然科学基金项目(61073197)资助;南京市科技计划项目(201608009)资助.

摘　　要：视频码率自适应是提高视频服务质量的一种有效方法.现有视频码率自适应算法大多都试图将一套相对固定的模型规则应用于所有用户,无法确保所有用户都拥有良好的QoE.针对上述问题,本文提出一种基于强化学习的自适应视频码率调节方案,可以根据不同的网络条件实时调整视频码率算法的参数.该方案通过强化学习方法,提高了学习收敛速度,限制了次优选择,并且不会因为快速收敛而降低视频码率调节的效率.实验结果表明,基于强化学习的自适应视频码率调节方案与Bola,M PC等传统视频ABR算法相比,总体平均比特率提高了大约8.3%,这得益于本文提出的方案能够更好地在不同的网络状态下对视频码率调节行为进行细粒度定制和优化.Video bitrate adaptation is an effective way to improve the quality of video services.Most of the existing ABR algorithms try to apply a set of relatively fixed model rules to all users,which often cannot ensure that all users have consistent high quality servicso the parameters of the algorithm must be based on the respective network throughput of each user.Aiming at the above problemthis paper proposes an adaptive video rate adjustment scheme based on reinforcement learning,which can adjust the parameters used in the algorithm in real time according to different network conditions.The scheme pre-calculates the optimal parameter settings of a given video rate adaptive algorithm under different network conditions,and then dynamically adjusts the parameters of the current algorithm at runtime.The experimental results show that compared with the traditional video ABR algorithm such as Bola and MPC,the overaverage bitrate is improved by about 8.3%,because it is better able to fine-grain and self-evolve video bitrate adaptive behavior in different network states.

关 键 词：码率自适应算法 强化学习 码率控制 QOE 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]