基于Spark Streaming的快速视频转码方法  

Fast video transcoding method based on Spark Streaming

作　　者：付眸 杨贺昆 吴唐美 何润 冯朝胜[1,2] 康胜 FU Mou;YANG Hekun;WU Tangmei;HE Run;FENG Chaosheng;KANG Sheng(School of Computer Science, Sichuan Normal University, Chengdu Sichuan 610101, China;Visual Computing & Virtual Reality Key Laboratory of Sichuan Province ( Sichuan Normal University ) , Chengdu Sichuan 610101, China;Sichuan Normal University Technology Park Development Company Limited, Chengdu Sichuan 610066, China)

机构地区：[1]四川师范大学计算机科学学院,成都610101 [2]可视化计算与虚拟现实四川省重点实验室(四川师范大学),成都610101 [3]四川师大科技园发展有限公司,成都610066

出　　处：《计算机应用》2018年第12期3500-3508,共9页journal of Computer Applications

基　　金：国家自然科学基金资助项目(61373163);国家科技支撑计划项目(2014BAH11F02;2014BAH11F01);四川省科技支撑计划项目(2015GZ079)~~

摘　　要：针对单机视频转码方法转码速度较慢和面向批处理的并行转码方法效率提升有限的问题,基于Spark Streaming分布式流处理框架,提出了一种面向流处理的快速视频转码方法。首先,使用开源多媒体处理工具FFmpeg,构建了自动化的视频切片模型,提出编程算法;然后,针对并行视频转码的特点,对弹性分布式数据集(RDD)进行研究,构建了视频转码的流处理模型;最后,设计视频合并方案,将合并后的视频文件进行有效储存。根据所提出的快速视频转码方法设计与实现了基于Spark Streaming的快速视频转码系统。实验结果表明,与面向批处理Hadoop视频转码方法相比,所提方法转码效率提升了26. 7%;与基于Hadoop平台的视频并行转码方法相比,该方法转码效率提升了20. 1%。Aiming at the problems of slow transcoding speed of single-machine video transcoding method and limited efficiency improvement of parallel transcoding method for batch processing,a fast video transcoding method for stream processing based on Spark Streaming distributed stream processing framework was proposed.Firstly,an automated video slicing model was built by using the open source multimedia processing tool of FFmpeg and a programming algorithm was proposed.Then,in view of the characteristics of parallel video transcoding,the stream processing model of video transcoding was constructed by studying Resilient Distributed Datasets(RDD).Finally,the video merging scheme was designed to store the combined video files effectively.Based on the proposed fast video transcoding method,a fast video transcoding system based on Spark Streaming was designed and implemented.The experimental results show that,compared with the Hadoop video transcoding method for batch processing,the proposed method has improved the transcoding efficiency by26.7%,and compared with the video parallel transcoding based on Hadoop platform,the proposed method has improved the transcoding efficiency by20.1%.

关 键 词：视频转码 SparkStreaming 分布式流处理 FFMPEG 弹性分布式数据集 

分 类 号：TP301.6[自动化与计算机技术—计算机系统结构]