三维投影变换辅助的云游戏编码优化算法  

Cloud Gaming Video Coding Optimization Algorithm Assisted by Three-Dimensional Projection Transformation

作　　者：王一帆 王昊 孙浩楠 尘原 霍俊彦[1] 王豪 张伟[1,3] WANG Yifan;WANG Hao;SUN Haonan;CHEN Yuan;HUO Junyan;WANG Hao;ZHANG Wei(School of Telecommunications Engineering,Xidian University,Xi’an 710071,China;Alibaba Group,Hangzhou,310000,China;Artificial Intelligence Research Center,Peng Cheng Laboratory,Shenzhen,Guangdong 518055,China)

机构地区：[1]西安电子科技大学通信工程学院,西安710071 [2]阿里巴巴集团,杭州310000 [3]鹏城实验室人工智能研究中心,广东深圳518055

出　　处：《西安交通大学学报》2023年第12期20-29,共10页Journal of Xi'an Jiaotong University

基　　金：国家自然科学基金资助项目(62101409,62171353)。

摘　　要：针对传统视频压缩技术无法对云游戏视频中复杂的全局相机旋转以及剧烈变化的大范围运动进行很好的运动估计与补偿的问题,提出了一种三维投影变换辅助的编码优化算法,以提高云游戏视频的编码效率。分析了三维游戏场景下一点投影到二维平面的详细空间变换过程,在此基础上通过建立约束进行最优化估计构建了虚拟相机的内参估计算法。利用估计的内参,结合相机运动信息,基于图像单应性理论生成了与待编码帧平行的辅助参考帧,利用其更新参考帧列表参与编码。由于生成的辅助参考帧与待编码帧之间只有平移运动,所以更有利于传统的运动估计和补偿。实验结果表明:与原始的高效视频编码(HEVC)相比,所提算法在云游戏视频上可以实现平均8.14%的BD-rate增益,一定程度上缓解了云游戏对网络带宽的高需求;特别地,在仅有相机旋转的情况下,BD-rate可达-24.68%,同时编码时间平均缩短5.34%;与HEVC-SCC相比,所提算法的BD-rate性能增益超过20%。For the problems that traditional video compression techniques are poor in motion estimation and compensation for complex global camera rotation and abrupt large-scale motion in cloud gaming videos,a coding optimization algorithm assisted by the three-dimensional projection transformation was proposed to improve the coding efficiency for cloud gaming videos.Firstly,the detailed spatial transformation process of projecting a point from a three-dimensional gaming scene to a two-dimensional surface was analyzed.Then,an intrinsic parameter estimation algorithm for the virtual cameras was constructed base on the analysis by establishing constraints for optimal estimation.With the estimated intrinsic parameters and the camera motion information,an auxiliary reference frame parallel to the frame to be coded was generated based on image homography theory and used to update the reference frame list for coding.As the generated auxiliary reference frame only involves translational motion with the frame to be coded,it is more conducive to traditional motion estimation and compensation.The experimental results show that,compared with the original HEVC,the proposed algorithm can achieve 8.14%BD-rate gains for cloud gaming videos on average,relieving the high demand for network bandwidth of cloud gaming to some extent.In particular,the BD-rate can be up to-24.68%in the case of only camera rotation.At the same time,the average coding time is reduced by 5.34%.In addition,compared with HEVC-SCC,the proposed algorithm can also achieve the BD-rate gains of more than 20%.

关 键 词：云游戏编码 三维投影变换 内参估计 参考帧生成 

分 类 号：TN911.7[电子电信—通信与信息系统]