基于Spark Streaming的视频/图像流处理与新的性能评估方法  

A novel performance evaluation method for processing video/image stream based on Spark Streaming

作　　者：黄文辉[1] 冯瑞 

机构地区：[1]复旦大学计算机科学技术学院,上海201203 [2]上海视频技术与系统工程研究中心,上海201203

出　　处：《计算机工程与科学》2015年第11期2055-2060,共6页Computer Engineering & Science

基　　金：国家科技支撑计划(2013BAH09F01);上海市科委科技创新行动计划(14511106900)

摘　　要：智能视频监控技术在公共安全、交通管理、智慧城市等方面有着广泛的运用前景,需求日益增长。随着摄像头安装的数量越来越多,采集的图像数据量越来越大,靠单台计算机处理已经远远不能满足需求了。分布式计算的兴起与发展为解决大规模的数据处理问题提供了很好的途径。使用一种基于Spark Streaming的视频/图像流处理的测试平台,阐述了平台的构成和工作流程,深入研究各个参数对集群性能的影响,创新性地提出了CPU时间占用率作为性能评估指标,与总的处理时间结合,更为全面反映集群性能和资源利用率。Intelligent video surveillance technology has a promising application prospect and growing demand in public safety, traffic management, smart city, etc. With a growing number of cameras used in video surveillance, the amount of image data collected by cameras is becoming bigger and bigger, which is out of the processing capacity of one single machine. The rise and development of distributed computing provides a good way to solve the problem of big data processing. We introduce a testing plat- form based on Spark Streaming, which is used to process video/image data received as stream, and illustrate the composition and working process of the platform. The impact of several important parameters on the performance of the cluster is deeply studied. In particular, the time-occupancy-rate of the CPU is initially proposed as one of the performance evaluating indicators, and together with the total processing time, it demonstrates the performance and resource usage of clusters more comprehensively.

关 键 词：SPARK STREAMING CPU时间占用率 分布式 视频/图像处理 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]